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Executive Summary 



The National Commission for Employ- 
ment Policy (NCEP) contracted with the In- 
stitute for the Study of Adult Literacy at Penn 
State to develop two evaluation designs for 
computer-assisted instructional packages 



used in the remediation cf basic skills in the 
Job Training and Partnership Act (JTPA) 
programs. Basic skills include reading, com- 
putation, communication, problem solving, 
and analysis. 



Background 



The use of computer-assisted instruction- 
al (CAI) packages for the remediation of 
basic skills in employment training programs 
is increasing rapidly. The "basic skills 
crisis," the national need to increase basic 
skills learning efficiency, and tte federal 
response of focusing employment and train- 
ingresources on the assessment andremedia- 
tion of basic skills have contributed to the 
proliferation of use of CAI packages in 
employment training programs. Consequent- 
ly, employment and training professionals 
are concerned about the effectiveness and 
efficiency of CAI packages are requesting 
assistance in obtaiiing reliable information 



about the utility of CAI packages in the 
remediation of basic skills and in identifying 
effective procedures for selecting ap- 
propriate and effective educational software. 

This project addressed the needs articu- 
lated by the employment and training profes- 
sionals with the production of five important 
products: a rationale for utilization of CAI 
packages with JTPA clients; a taxonomy 
based on client characteristics and technol- 
ogy features; a survey-type form for evaluat- 
ing CAI packages; and two evaluation 
designs, a large-scale, longitudinal study 
and a small-scale survey. 



Project Methodology 



The rationale for the use of CAI packages 
in JTPA programs was based upon client 
characteristics and salient features of com- 
puter technology. Taxonomical features 
were culled from a review of relevant litera- 
ture identifying principal factors and issues 



germane to the evaluation designs, a series of 
site visits permitting interviews with project 
personnel and observations of operating 
programs, and structured interviews with 
members of expert panels, consultants, ser- 
vice providers, and vendors. Written and ver- 
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bai responses to the rationale and taxonomy 
from more than 35 respondents were incor- 
porated into the evaluation designs. 

Panels of representatives from the U.S. 
Departments of Labor and Education, the 
AFL-CIO, business and industry, minority 
groups, and Adult Literacy and Technology 
(ALT) consultants, were formed to review 
and critique project materials as they 
developed A total of 15 panelists reviewed 
materials and made contributions during the 
developmental stage of the evaluation 
designs. 



Five JTPA sites, four located in Pennsyl- 
vania and one in New Jersey were visited by 
project staff. These sites, representing a 
range of program types, provided important 
data about the actual utilization of computer- 
assisted packages with JTPA clients. Addi- 
tional service providers were surveyed by 
telephone. 

CAI packages published by 12 different 
vendors were provided for review and ex- 
amination. Additionally, a site visit was 
made to a Delaware software company to 
pilot test the evaluation form on CAI pack- 
ages designed for JTPA clients. 



Taxonomy of CAI Package Features 



The taxonomy developed by this project 
is a system of classifying CAI packages 
based on the needs of the JTPA population 
and the degree of complexity and quality of 
the technology employed in the CAi pack- 
ages used by JTPA programs in the remedia- 
tion of basic skills. The taxonomy requires 
consideration of three categorical factors 
useful in describing CAI packages: 

1.0 Types of CAI Packages 

LI Purpose of the Package 

1.2 Context Orientation'of the Package 

L3 Instructional level(s) of the 
Materials 

It also requires consideration of three 
major factors affecting quality of CAI pack- 
ages intended to be used in JTPA programs: 



2.0 Quality Measures 

2.1 Curriculum Features 

2.2 Software Design Features 

2.3 Implementation Features 

The taxonomy includes a description of 
various aspects of each of these six factors 
and reflects the concern of educators and 
trainers for giving increased attention to the 
curriculum and related learning activities 
provided through the CAI packages. The cur- 
riculum is a major factor in stimulating and 
guiding student learning. Ii is important, 
therefore, that CAI packages be evaluated 
not only on their technical merits in terms of 
software design and implementation options 
but also on their curriculum design. 

An evaluation instrument has been 
designed to establish values for each of the 
factors in the taxonomy. The instrument 
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provides an objective means of categorizing 
CAI packages by producing standardized 
numeric measures of quality for each of the 
features identified by the taxonomy. 

The evaluation instrument, together with 
the taxonomy, will serve the important role 
in the evaluation designs of operationalizing, 



establishing values for, several of the impor- 
tant variables related to the CAI packages 
used in JTPA programs. They will also pro- 
vide a means of obtaining a uniform, objec- 
tive rating for CAI packages intended for use 
in JTPA programs that may have utility out- 
side of the planned evaluation process. 



Evaluation Designs 



Two evaluation designs have evolved 
from the rationale and taxonomy: (a) a large- 
scale, longitudinal design equal to ap- 
proximately six professional person years of 
effort; (b) a small-scale, survey type evalua- 
tion efforu Both evaluation designs are struc- 
tured to provide information about the 
effectiveness and efficiency of computer-as- 
sisted instructional (CAI) packages used in 
the remediation of basic skills in JTPA 
programs. The primary purpose is to obtain 
information about the differential effects of 
CAI packages and their cost-effectiveness 
rather than to evaluate the participating 
programs per se. 

The criterion variables used in both are 
measures of JTPA participant performance 
on standardized measures of achievement in 
the basic skills. The effectiveness of the pro- 
gram in delivering instruction to participants 
is assessed by pre-test and a post-test 
measures of their performances on stand- 
ardized measures of achievement in the basic 
skills. The primary question to be addressed 
is the extent to which the use of CAI pack- 
ages contribute to increased JTPA par- 



ticipant performance in basic skills com- 
pared with the increase provided by tradi- 
tional methods of instruction, those not using 
CAI packages in the remediation of basic 
skills. 

The large-scale evaluation design has an 
expected duration of two years and involves 
a variety of methods employed to assess the 
effectiveness and efficiency of CAI packages 
used in the remediation of basic skills in job 
training programs. It evaluates and contrasts 
two treatment modes: computer-assisted in- 
struction against traditional instruction, each 
in their natural program settings. The design 
uses existing program participants, instruc- 
tors, teaching methods, curriculum 
materials, software, and equipment and at- 
tempts to monitor and measure the differen- 
tial effects of the treatment modes. 

The small-scale design followed prin- 
cipally the same analytic framework but 
relies on surveys for data collection. Both 
designs use an evaluation form developed in 
the project to assess the quality of CAI pack- 
ages used in the remediation of basic skills. 
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Introduction 



The use of computer-assisted instruction- 
al (CAI) packages for the remediation of 
basic skills in employment training programs 
is increasing rapidly. The "basic skills 
crisis," the national need to increase basic 
skills learning efficiency, and the federal 
response of focusing employment and train- 
ingresources on the assessment and remedia- 
tion of basic skills have contributed to the 
proliferation CAI package usage in employ- 
ment training programs. Consequently, 
employment and training professionals are 
concerned about the effectiveness and ef- 
ficiency of CAI packages and are requesting 
assistance in obtaining reliable information 
about the utility of CAI packages in the 
remediation of basic skills. The information 
is needed in making decisions involving the 
use or expanded use of CAI packages in their 
programs and in identifying effective proce- 
dures for selecting appropriate and effective 
educational software. 

The National Commission for Employ- 
ment Policy (NCEP) contracted with the In- 
stitute for the S tudy of Adult Literacy at Penn 
State to develop two evaluation designs for 



investigating the effect of computer-assisted 
instructional packages used in the remedia- 
tion of basic skills m the Job Training and 
Partnership Act (JTPA) programs. Basic 
skills include reading, computation, com- 
munication, problem solving, and analysis. 

A number of tasks were included in the 
scope of work assigned to the evaluation 
design project, including a survey of current 
knowledge about the key elements involved 
in the evaluation: training programs, par- 
ticipants, staff, CAI packages, and current 
research. 

A significant task was to develop a 
taxonomy of features found in CAI packages 
based on the needs of the targ population 
and on the degree of complexity and quality 
of the technology employed The taxonomy 
provided the rationale for an instrument to 
evaluate CAI packages. The two evaluation 
designs which evolved present alternative 
approaches to investigate CAI effectiveness 
at different levels of effort: (a) a large, lon- 
gitudinal study equal to approximately six 
professional person years and (b) a small- 
scale, survey type project. 
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Chapter 1 

Survev of Current Knowledge 

4/ 



The project undertook a survey of current 
knowledge relative to the characteristics of 
JTPA programs and participants and com- 
puter-assisted instruction. The survey was 
accomplished with a review of relevant 
literature, a series of site visits allowing in- 
terviews with projec* personnel and observa- 
tions of operating programs, and input from 
members of expert panels, service providers, 
and vendors. 

The intent of the literature review was to 
obtain and present a clear understanding of 
the principal factors and issues related to the 
prospective evaluation designs, rather than to 
produce a comprehensive review of the 
literature on any single topic. 

Five JTPA sites visited by project staff, 
four located in Pennsylvania and one in New 
Jersey, represented a range of program types. 
The site visits provided important data about 
the actual utilization of computer-assisted 
packages. The interviews andobservations at 
the sites yielded descriptive information 



about the characteristics of classrooms, par- 
ticipants, and instructors. 

Valuable information was provided by an 
expert panel composed of representatives 
from various organizations nationwide. Dr. 
Eunice N. Askov, Director of the Institute for 
the Study of Adult Literacy at Penn State, 
extended invitations to participate to repre- 
sentatives of theU. S. Departments of Educa- 
tion and Labor, AFL-CIO, business and 
industry, and minority groups. Panelists and 
Adult Literacy and Technology (ALT) con- 
sultants were each sent a survey question- 
naire, a project abstract, and drafts of 
pi?Uminary materials developed for use in 
subsequent evaluation designs. The repre- 
sentatives were asked to respond in writing 
or by telephone. 

Contacts were made with service 
providers nationwide through their respec- 
tive service delivery areas (SDAs) or upon 
recommendations froni members of the ex- 
pert panel and vendors. Chosen because of 
geographic location, demographic repre- 
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sentation, CAI utilization, and auessability, 
service providers were contacted by 
telephone and interviewed, guided by the 
survey questionnaire. Those wishing to par- 
ticipate further were sent additional materials 
to review and critique. 

Time constraints posed problems with 
mail turn-around and telephone contacts, 
thus limiting the number of respondents. A 
listing of all participants is provided in the 
appendix. 

As anticipated, the information gathered 
was invaluable in the development of the two 



evaluation designs. The responses from the 
expert panel, ALT consultants, service 
providers and vendors, incorporated in this 
report and in the evaluation designs, 
provided input on four key issues: 

• Characteristics of JTPA programs 

• Characteristics of JTPA participants 

• CAI use in JTPA programs 

• Cost-effectiveness of CAI packages 



Characteristics of Job Training and Partnership Act 

Programs 



The JTPA operates under the block grant 
concept of funding state and local programs. 
Primary responsibility for administering the 
federally funded programs rests with the 
states. States assume the oversight and ad- 
ministrative functions while local com- 
munities undertake actual delivery of 
program services to participants. Funds are 
distributed to state governors according to a 
formula specif ied in the legislation based on 
unemployment and poverty figures. The 
governors reallocate 78 percent of the money 
by similar formula to SDAs, approximately 
600 nationally, within their states. SDAs sub- 
sequently reallocate funds to local projects. 

JTPA operates on forward-funding in- 
tended to reduce uncertainties and problems 
associated with late appropriations. Program 
planners should therefore know in advance 
funding levels available for the year. Further, 
the states are guaranteed to receive no less 
than 90 percent of their preceding fiscal 
year's allocation from the block grant pro- 



gram. These funding procedures are very 
important for programs considering the pur- 
chase of computer equipment and implemen- 
tation of a computer-assisted instructional 
component The overall funding level for 
JTPA has been stable at approximately $3.5 
billion per year since it began in October, 
1983 (NCEP, 1987, pp. 5-8). 

An examination of the characteristics of 
JTPA programs involves a categorization by 
title. Each title addresses different objectives 
for servicing the needs of distinct target 
populations. 

Title II: Adult and Youth 
Programs 

Eligibility for Title II-A programs is 
broadly defined by the legislation; par- 
ticipants need only be "economically disad- 
vantaged M to receive services. A 1985 
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demographic report described the 752,900 
people served as (NCEP, 1987, p. 53): 

• 92% economically disadvantaged 

• 53% female 

• 45% minority 

• 42% 21 years or younger 

• 40% public assistance recipients 

Program services are divided into five 
categories: 

1. Classroom training (CT): basic 
education, occupational skills 
training, or a combination of the 
two 

2. On-the-job training (OJT): skills 
training in a specific occupation in 
an actual work setting 

3. Job Search Assistance (JSA): sup- 
port with locating, applying for, or 
obtaining jobs. 

4. Work Experience (WE): part-time 
or short-term subsidized employ- 
ment designed to assist participants 
in entering/re-entering the labor 
market. 

5. Other services: those not included 
within the other categories. 

The performance of JTPA programs is 
monitored with national performance stand- 
ards established by the Secretary of Labor. 
The performance standards reflect the 
"return on investment" model in which in- 
vestment in the programs s assessed against 



increased participant employment and earn- 
ings and decreased welfare dependency. In 
1986, the average cost-per-placement stand- 
ard was $4,374 for adults and $3,711 for 
youth (NCEP, 1987, p. 12) 

Title II-B provides for summer programs 
serving disadvantaged youth aged 14 to 21. 
Funds may be used for remedial and basic 
education, as well as WE, OJT, or any job- 
related activities which prepare youth for 
work. The 1987 cost was approximately 
$1,000 per participant (NCEP, 1987, p. 82). 



Title III: Employment and 
Training Assistance for Dislocated 
Workers 



Dislocated workers are individuals whose 
jobs have been terminated because of plant 
closure, unacceptable performance, or other 
reason for which they are unlikely to return, 
or who are long-term unemployed with 
limited prospects for re-employment in pre- 
vious occupations. Nearly 170,000 par- 
ticipants were served from the beginning of 
the program in October,1983, through 
March, 1985 (NCEP, 1987,p. 95)and95,600 
in program year 1985 (p. 219). 

Participant characteristics reported by the 
U. S. Department of Labor and General Ac- 
counting Office (GAO) in 1987 were 
(NCEP, 1987, p. 93): 

• 60% male 

• 69% white 

• 88% between the ages of 22-44 

• 55% high school graduates 

• 60% from the manufacturing sector 
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9 57% receiving unemployment insurance 

Program services provide basic skills 
training, CL OJT, and job placement assis- 
tance. 

Older, less educated dislocated workers 
are reportedly less likely to participate in the 
program, possibly because of apprehensions 
about remedial or classroom situations- 
Potential clients in this category have dif- 
ficulty meeting minimum qualifications for 
training programs and are thus more likely to 
be victims of aie "creaming" effect in which 
only clients with the best prospects for suc- 
cess are selected for services. 



title IV: Federally Administered 
Programs 

The major program under this title is the 
Job Corps. Native Americans, migrant and 
seasonal farm workers, and veterans are also 
entitled to receive services under separate 
programs. 

The purpose of Job Corps is to assist 
economically disadvantaged young men and 
women, aged 14 to 22, who can benefit from 
an intensive residential program. The dura- 
tion of the total service programis usually six 
months and includes remedial education, 
skills training, and work experience. Par- 
ticipants are provided subsistence, clothing, 
health care, and recreation. Currently, 107 
Job Corps programs service 100,000 persons 
per year at an estimated cost of $15,000 per 
pcison. 



Characteristics of JTPA Participants 



JTPA participants can be categorized as 
h : gh school dropouts, low-achieving high 
school graduates, displaced homemakers, 
dislocated workers, and English-as-a- 
Seccnd-Language (ESL) clients. Obviously, 
the needs and learning styles of such a 
diverse group will vary greatly. 

Needs 

A high percentage of adults with learning 
disabilities have been recognized by teachers 
in Adult Basic Education (ABE) programs, 
according to Ross and Smith (1988). Travis 
(1979) suggested that the incidence may be 
as high as 80 percent, a sharp contrast to the 



learning disabled (LD) population, reported- 
ly less than 5 percent, in public schools. ABE 
teachers in the Ross-Smith study concluded: 

All groups acknowledge that poor writing and 
spelling skills vere problems frequently faced 
by learning disabled students, but they also 
believed that LD adults frequently find ways 
to compensate for their learning problems. 
Whenaskedwhetherthey felt that skill deficits 
in adults with learning disabilities were caused 
by poor study habits, they disagreed. Respon- 
dents also disagreed that support services for 
the learning disabled tend to delay develop- 
ment of self-reliance and independence, but 
did not believe that any program for learning 
disabled students would be sufficient to 
eliminate learning problems, (p, 25) 
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Mikulecky, Ehlinger and Meenan (1987) 
predicted only a small and shrinking number 
of available jobs in which workers could 
function with low level literacy skills* Newly 
created jobs require reading and writing 
skills considerably higher than the basic 
skills level. The demand is for "broad" tech- 
nicians rather than "high" technicians 
(Mikulecky et al., 1987). Therefore, adults in 
JTPA programs will have few occupational 
choices if they cannot meet the high literacy 
requirements of the work place (Mikulecky 
etal., 1987). 



Learning Styles 



No specific research on the learning styles 
of JTPA participants has been documented 
Because, the profiles of the typical drop-out 
and low achiever, constituents highly repre- 
sented within the JTPA population, indicate 
difficulty with processing information, the 
following research on learning styles can be 
justifiably applied to JTPA participants in 
general. 

Learning style has been defined by Bot- 
trorf-Hawes (1988, p. 40) as: 

a person's modality strength (auditory, visual, 
or kinesthetic-tactual). combined with the 
brain hemispheric preference for organizing 
and processing information (right/global; 
left/sequential; mixed). 

The growing body of knowledge about 
learning styles and the cerebral processing of 
information mandates a change in the tradi- 
tional methods of teaching* Bottroff-Hawes 
maintains that schools are unresponsive to 
half of the school population because of the 
proliferation of traditional instructional ac- 
tivities best suited for left-brain dominant, 
auditory students. 



Dixon ( 1 982) emphasized the necessity of 
considering learning styles of participants 
with other factors when designing a training 
program. Because the dominant learning 
style dictates the way an individual processes 
and gathers information, Dixon maintained, 
accommodating learning style differences 
can improve participant learning and reduce 
training time (Dixon, 1981). Likewise, 
Dunn and Bruno (1985) demonstrated in 
school situations that accommodating stu- 
dent learning styles improved performance. 

No single learning style is superior to any 
other, Martin (1986) concluded; most stu- 
dents use all learning styles to some degree. 
However, because most students use their 
dominant learning style at least 90 percent of 
the time (Keefe, 1979), Martin emphasized 
that teacher understanding of learning styles 
can provide students with optimum learning 
experiences. 



Respondent Comments 



The information received from the 
panelists, consultants, and service providers 
addxes^eJ the needs and learning styles of 
JTPA participants. The responses may be 
divided into the categories of the needs for 
youth, adults, adult work force entrants, dis- 
placed workers, and non-English speaking 
adults. 

Youth 

• Basic skills including reading, math, 
computer literacy skills, pre-employ- 
ment skills, personal development, 
hygiene, and nutrition 
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• General Equivalency Diploma (GED) 
and occupational information and train- 
ing 

• Development of basic skills so that a 
sense of accomplishment is provided for 
students 

• Supportive services for disabilities 

• Screening for developmental disabilities 

• More individualized instruction and 
coaching 

• Hands-on-learning experiences 

• Counselors for goal-setting, motivation, 
and building self-esteem 

• Assistance with making connections be- 
tween training and work situations 

• More variety in activities <md shorter seg- 
ments thai; adults to counteract hyperac- 
tivity 

• Assistance with problems associated 
with single-parent families, rebellious- 
ness, peer pressure, and offenders 

• More exposure to diverse settings, situa- 
tions, and opinions 

• Financial assistance 

• Child care 

• Transportation 

AduU 

• Upgrading reading and math skills 



• Day care and other support services, such 
as transportation and personal develop- 
ment skills 

• Entry-level job skills 

• Age-appropriate materials 

• Assistance with accommodating for 
severe learning difficulties which many 
have 

• Pre-training sessions for use of JTPA 
progiams 

• GED preparation and examination 

• Awareness of choices and possibilities 

• Improving concentration and focusing 

• Survival and coping skills 

• Building self-esteem and overcoming 
feelings of failure and inadequacy 

• Assurance in new learning situations 
Adult Work Force Entrants 

• Processing and organizing information 
"to do" and "to access" 

• Basic skills 

• Employment training 

• Knowledge of personal strengths and 
weaknesses as related to different job 
requirements 

• Assistance with changing negative at- 
titudes and reacting to adverse situations 

• Improving work habits 
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• Accessing telephone and transportation 
services 

Displaced Workers 

• Assistance with transferring work ex- 
perience from one job to another 

• More relevance and immediacy of ap- 
plication of skills 

• Improving job skills and job search tech- 
niques 

• Upgrading skills in math and reading 

• Working through the anger and depres- 
sion of job loss 



- Building self esteem 
Non-English Speaking Adults 



English language training 

Assessment to determine other learning 
difficulties 

Remedial education in math and reading 

Visual, auditory, and kinesthetic aids 

Job skills training and job search tech- 
nique 

Child-care 

Transportation 



Computer Assisted Instructional Packages for JTPA 

Programs 



The literature review identified salient 
features for creating a taxonomy of CAI 
evaluation criteria to be used in the evalua- 
tion designs. The criteria were developed 
from numerous studies of CAI usage in train- 
ing and educational programs. 



Use of Computer-Assisted 
Instruction in Training Programs 

CAI packages continue to attract attention 
among training professionals (Kamouri, 
1984). Declining costs of computers and 
technical advances in software and hardware 
design have increased the possibility for in- 



dividualizing and improving the training 
process for people at different levels of 
ability (Kamouri). As illiteracy approaches 
crisis proportions, the need exists for educa- 
tional programs capable of increasing the 
probability of learner mastery and cor- 
responding more closely with the world of 
work (Caldwell & Hedl, 1984). 



General 

Clients in a successful project evaluated 
by Caldwell and Hedl (1984, p. 77): 

1. Indicated that the computer-as- 
sisted instruction made reading 
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easier, particularly because the 
computer offered greater amounts 
of reinforcement and instruction on 
an individual basis than the teacher 
could in a typical classroom situa- 
tion 

2. Expressed apreference for learning 
via the computer rather than in a 
class where their learning deficien- 
cies were easily recognized by 
teachers, and mora significantly, 
by other learners 

3. Favorably changed their opinions 
about reading and working with 
numbers 

4. Demonstrated an increased interest 
and self-motivation in regard to 
learning new things 

5. Expressed more self-confidence 
about success in testing situations 

Simulations 

A study by Bruhn, Grabenhofer, and 
Cather (1984) of CAI simulation in industry 
indicated savings in instructor and student 
time and improved instruction with self- 
spaced andreality-based learning. The system 
maintained student records, eliminated need 
for instructor observation, and reduced need 
for individual feedback. This type of instruc- 
tion eased peer pressure and increased access 
to varied and individualized practice. 



"Supplemented" CAI 

Kamouri (1984) examined "supple- 
mented" CAI training systems by applying 
the same standards and measurements. Sup- 
plemented CAI programs use adjunct modes, 
such as highly interactive graphics, 
simulators, and applied exercises, that allow 
trainees to apply concepts and to perfect 
skills within simulated environments. The 
majority of research incorporating supple- 
mented CAI has been supported by the 
military because of their intense training 
needs. 

Displaced Workers 

The use of CAI is increasing with low 
literate adult learners. A study by the Institute 
for the Study of Adult Literacy at Penn State 
supports the utilization of CAI with dis- 
placed workers (Bixler and Askov, 1988). 
Functionally illiterate workers are un- 
employed for longer periods of time, are less 
likely to qualify for job training programs, 
and less likely to participate in traditional 
basic skills programs. Traditional literacy in- 
struction utilizes a tutor to deliver instruc- 
tion. Because computers in the work place 
are common, Bixler and Askov emphasize 
the importance of the acquisition of both 
basic skills and computer literacy skills to 
ensure competitiveness in the marketplace. 
Further, they stress teaching skills in a func- 
tional context. Prompt recognition of the 
relevance of learning is particularly powerful 
with this population. "By using the computer 
to teach basic skills, the student is also learn- 
ing computer skills as well" (p. 16). 
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CAI versus Traditional 

Students in Askov, Maciay, and Bixler's 
1987 CAI study also demonstrated greater 
learning increases within a given period of 
time, in comparison to traditional methods. 
Clarke (1983) cites other studies with similar 
results. These researchers caution that lon- 
gitudinal studies should be conducted in 
order to confirm their findings. 

CETA 

Studies, however, in the use of CAI in 
CETA programs have reported somewhat 
negative findings. In two basic skills 
programs investigated by Caldwell and Hedl 
(1984), the CAI program was not significant- 
ly more effective than the regular program. 
Both programs were plagued by irregular 
attendance, lack of commitment to program 
goals, and failure to complete significant por- 
tions of the curricula. Caldwell and Hedl 
(1984) concluded that appropriate incentives 
for acquiring basic skills are necessary for 
success in both the teaching and learning 
process. 

The researchers identified conditions es- 
sential for success for any JTPA basic skills 
project (p. 84): 

1. Learners must want to acquire 
skills. 

2. Learners must have an accurate 
perception of what they are learn- 
ing. 

3. Learners must take responsibility 
for a task and accept the conse- 



quences of success or failure in per- 
forming it.- 

In addition, the study demonstrated the 
importance of a full-time teacher who is com- 
mitted to improving instructional perfor- 
mance. Computer-assisted materials must 
integrate job skills, life skills, and basic skills 
to make the learning important and relevant 
to life. 

Caldwell and Hedl argued against un- 
qualified acceptance of the negative findings 
in their study. T.iey advocated computer-as- 
sisted instruction as being particularly useful 
for teaching basic skills to adults who have 
experienced learning difficulties. 

• A multi-sensory approach to learning can 
be highly motivating. 

• The curriculum is individualized and 
feedback immediate. As a student 
progresses, confidence grows. 

• Programs are flexible and adaptable to a 
wide variety of needs. 



Use of Computer- Assisted 
Instruction in Schools 



Because many JTPA programs focus on 
the needs of youth, knowledge of the use of 
CAI materials in schools and their compara- 
tive strengths and weaknesses will enhance 
the present study. 

Cooperative Learning 

Johnson, Johnson, and Stanne (1986) 
compared the relative efficacy among 
cooperative, competitive, and individualistic 



12 



learning situations in promoting (a) achieve- 
ment, (b) oral participation by male and 
female students, (c) equal status between 
male and female students, and (d) positive 
attitudes toward computers. Seventy-four 
eighth grade students were randomly as- 
signed to conditions stratifying for sex arid 
ability. The study found that computer-as- 
sisted cooperative learning promoted the fol- 
lowing: 

• Higher quantity and quality of daily 
achievement 

• Accuracy of recognition of factual infor- 
mation studied and ability to apply facts 
in tt^i questions requiring higher-level 
reasoning and problem solving 

• More success in complex problem-solv- 
ing tasks involving mapping and naviga- 
tion 

• Greater success in operating a computer 
program. 

In addition, students in the cooperative 
and competitive groups scored higher on 
achievement tests than the students in the 
individualistic group. Further, the coopera- 
tive computer groups promoted more stu- 
dent-to-student interaction and equal status 
for all group members. Noteworthy is past 
research indicating female avoidance of 
computers and under-representation in com- 
puter studies. 

Individualization 

That learners vary in the amount of time 
needed to master a lesson, depending upon 
their aptitudes, experience, and motivation, 
is common knowledge. CAI can vary from 



full adaptation to no adaptation for the 
learner. Studies by Ros3 (1984), Merrill 
(1980), and Tenneyson et al. (1984) indi- 
cated the potential of individualizing the 
amount of instruction for increasing effective 
learning. 

Yang (1987), studying the use of CAI 
with individualized instruction, distin- 
guished between "individual" and "in- 
dividualized" instruction. The former treats 
one student at a time, while the latter 
describes the tailoring of instruction to ft the 
individual's unique learning needs. This dis- 
tinction is relevant to the needs of JTPA 
youth participants whose learning styles re- 
quire "individualized" instruction. In- 
dividualized instruction, suggest Bergan and 
Dunn (1976), is characterized by: 

• A broad array of educational objectives 
and extensive alternative content 

• A variety of instructional procedures, set- 
tings, and contexts 

• An extensive cross-indexing of cur- 
ricular objectives, materials, methods, 
and learning contexts 

• An extensive data base regarding the in- 
dividual student's interests, abilities, 
aspirations, optimum learning styles, 
long-range goals, and ambitions 

• An extensive cumulative record on each 
student regarding his past academic ac- 
complishments and academic records 

• A file of information regarding con- 
straints imposed by parental preferences 
and state and local requirements and 
restrictions imposed virtue of supply, 
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logistics, and administrative considera- 
tions* 

In addition, Merrill (1984) and Tennyson, 
Christensen, and Park (1984) described five 
parameters through which individualized in- 
struction progresses: 

• Instructional amount 

• Display time 

• Instructional sequence 

• Personal attention 

• Internal learning activities. 

Pacing 

Each learner requires a different amount 
of time for processing and storing informa- 
tion (Yang, 1987). Individualized learning 
speed is important for both efficiency and 
effectiveness of learning. Two types of 
pacing to consider in learning speed are ex- 
ternal pacing and self -pacing. Results from 
studies are mixed. 

Slow learners learn better through slow- 
paced instruction rather than self-paced in- 
struction. Self-pacing can lead to learner 
procrastination. More research is needed in 
this area before appropriate choices can be 
made for various learners. 

Time Savings 

CAI packages release teachers from the 
need to be constantly motivating students to 
master various materials. (Mevarech & Rich, 
1985). The benefit for teachers in this regard 
is more time for other instructional activities 
(Dalton&Hannafin, 1984). 



Sequence 

Instructional sequence is important so the 
structure matches student knowledge. Ideal- 
ly, adult learners should have control over 
choosing content. Concern over instructional 
sequence exists, however, because learners 
are not always familiar enough with the sub- 
ject content to know how best to control its 
sequence (Yang, 1987), thereby creating 
gaps and deficits in essential information. 

Reinforcement 

One of the perennial virtues of CAI has 
been its positive reinforcement for the 
learner. Yang (1987) advocates, along with 
researchers Ross (1984) and Gaynor (1981), 
the importance of varying the feedback ac- 
cording to specific learner needs. 

Slow learners. need immediate feedback, 
while high level learners require end-of-ses- 
sion feedback. 

Self-esteem 

Many young JTPA participants have not 
been successful in school and are, therefore, 
considered to be at-risk. Research on the 
effect of CAI programs upon student self-es- 
teem may provide helpful information in 
dealing with this troubled target group. 

In 1985, Mevarech and Rich found that 
students who participated in CAI math 
programs rated themselves significantly 
higher on self-concept in arithmetic achieve- 
ment than students exposed to the same 
material in a traditional instructional setting. 
Brown (1986) and Dalton and Hannafin 
(1984) found mastery of subject content and 
development of computer literacy to be 
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potential sources of positive affective 
development In addition, they found that 
external support, especially in the form of 
peer group reinforcement, appeared to con- 
tribute to a positive attitude. Clement (1981) 
observed that non-judgmental, neutral, and 
consistent reinforcement offered by the com- 
puter is an optimal reward situation. 

Finally, all the researchers mentioned 
above emphasized the learner's freedom 
from embarrassment, disapproval, and 
diminished status which can occur in class- 
room situations when a student's failure to 
perform correctly is public knowledge. With 
CAI packages, students are allowed to learn 
through trial and error in private at their own 
pace. 

Robertson, Ladewig, Strickland, and 
Boschung (1987) developed a study to 
evaluate the effects of CAI packages on stu- 
dent global self-esteem, an area previously 
unaddressed empirically. Over 1,000 stu- 
dents taking a home economics course par- 
ticipated in the study, a two-group pre-test 
and post-test experimental design. A sup- 
plementary CAI package was used as the 
experimental treatment Students in the ex- 
perimental group scored significantly higher 
on self-esteem than did students in the con- 
trol group. The researchers emphasized the 
findings that the CAI packages had potential 
for enhancing students' self-esteem. 

Behavior Disorders 

Recentresearch has extolled the virtues of 
CAI packages for students with behavior dis- 
orders, an. attribute relevant to JTPA 
programs because of its emphasis on youth 
who have been unsuccessful in conventional 
school settings. 



Students with behavior disorders (BD) 
students may be described as lacking motiva- 
tion toward academic and social success in 
school and success in chool-related tasks 
(Manion, 1986). These students can also be 
characterized as being unable to concentrate 
on tasks for a sustained period of time. Be- 
havior disorders cause cumulative academic 
problems for the students, as well as class- 
room management and control problems for 
teachers of these students (Manion, 1986). 

The connection between CAI packages 
and the educational needs of BD students is 
based on the contention that CAI can control 
the learning environment both quantitatively 
and qualitatively (Bowman, 1982; Chaffin, 
Maxwell, & Thompson, 1982). CAI can 
stimulate depressed interest in learning, in- 
crease persistence, attention, and personal 
involvement in task performance. Further, it 
purportedly promotes self-esteem and self- 
discipline (Manion, 1986). 



Needs 



Hannafin, Dalton, and Hooper (1987) 
outlined certain computer-related needs 
which are important for JTPA programs 
serving youth 14-21 years of age. 

1. A comprehensive computer- 
driven curriculum. Although the 
type, quality, and quantity of 
educational software has increased 
dramatically in the last ten years, 
this collection of software could be 
termed a "patchwork" curriculum 
because it differs so radically from 
one vendor to another. 
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2. Greater computer friendliness* 
Although computers have ad- 
vanced significantly in this area, 
there is still room for improvement. 

3- Experiences and opportunities 
across economic and gender 
lines. Wealthy school districts are 
able to provide CAI simply because 
they can afford it Females tend to 
have more limited exposure to 
computers because of the percep- 
tion that computers are largely 
tools used in math and science 
courses in which females have 
traditionally been under-repre- 
sented 



Components of Effective CAI 
Instruction 



It is imperative that computer training 
systems be based upon theoretical founda- 
tions of learning (Kamouri, 1984). For- 
tunately, current research in cognitive 
psychology has significantly influenced their 
design. In recent years, the movement to 
integrate training and instructional thinking 
with the advances of cognitive science has 
gained momentum. Kamouri defines certain 
components of an effective instructional sys- 
tem that can be used for analysis. 

Responsiveness 

Responsiveness reflects the degree of 
learner-computer interaction and the adap- 
tability of the program to differences in skill 
or knowledge level across the learner popula- 
tion. The greater the inclusion of these fac- 



tors in the system design, the more respon- 
sive the system is, and the better its training 
potential. Kamouri cites numerous studies 
that lend credence to the idea that superior 
and rapid adult learning is associated with 
more personalized, self-paced, and self- 
directed computerized instruction. 

Flexibility 

Flexibility or user control, another related 
system component, is the ability to personal- 
ly vary the sequence of information presen- 
tation. This includes the individual user's 
determining the route through lessons, 
whether to use branching optic 'is, and the 
amount of time spent on different training 
tasks. Superior learning is reportedly found 
in highly flexible systems where the trainee 
has control over his/her own learning (Mc- 
Cann, 1981;Lehey, 1981). 

Multiple Modes 

The use of multiple modes in an instruc- 
tional program may also contribute to effec- 
tive instruction. Graphic displays, 
simulations and problem solving exercises 
can enhance the acquisition of concepts and 
facilitate the development of a more detailed 
model of the knowledge domain (McCann, 
1981). Graphics is one of the more actively 
researched components of instruction. 
Gagne (1981) determined graphics to be ef- 
fective as perceptual organizers in focusing 
attention to key features and important 
aspects of an information display. McCann 
(1981) cautioned that differences exist be- 
tween individuals of higher mental ability 
and individuals of lower mental ability. 
Higher mental ability individuals benefit 



16 



from presentations that are perceptually 
complex, informationally laden, of fixed 
pace, and with multi-channel motion pic- 
torial forms. The opposite is true of lower 
mental ability individuals. Therefore, CAI 
programmers should limit the animation and 
displays in presentations for slow learners. 



Measures of CAI Effectiveness 



An analysis of computer-assisted training 
research by Kamouri '1984) produced a 
number of significant findings. Various com- 
puter-assisted instructional systems were 
rated in cost efficiency, training time savings, 
managerial satisfaction, positive trainee at- 
titudes, and measures of objective perfor- 
mance on training related tasks. 

Training Time 

CAI research was examined for evidence 
of the measures identified by Kamouri. 
Training time savings was reported as a 
measure of effectiveness, accounted for by 
the self-pacing and individual scheduling 
flexibility features. Studies of electronic 
technicians and job related technical infor- 
mation (Schwartz and Haskell, 1966; 
Schwartz and Long, 1967) report significant 
savings of 10 percent and 15 percent respec- 
tively by computer instructed groups versus 
self-study groups and in centrally located 
computer-trained versus remotely trained. 

Very rapid success by a group whose CAI 
packages included options for review, recap, 
quiz, and rote learning was reported by 
Seidel, Wagner, Rosenblatt, Hillelsohn, and 
Stelzer (1978). The group was not empirical- 
ly compared to a non-CAI group, however. 
Krupp (1972) evaluated Honeywell 



Corporation's program to upgrade the skills 
of technical employees and facilitate the ac- 
quisition of new job related information. The 
interactive tutorial CAI system, responsive 
and adaptive to user input and flexible in 
ability to set the lesson branching networks 
according to user needs, averaged seven 
hours, in comparison to the 24 to 30 hour 
range for traditional instructional. 

Negative findings were reported by Wal- 
lis (1964) in a Naval Technical Training Sys- 
tem using a linear programmed instruction 
program with electrical mechanics. No sig- 
nificant decrease in training time compared 
with the traditional training system was indi- 
cated. 

In a follow-up study integrating the 
programmed instruction course with class 
tutorial sessions and prescribed text notes, 
Wallis (1964) reportedless training time than 
that for both the original programmed in- 
struction system and traditionally trained 
groups. The results indicate that CAI 
programs incorporating user control and 
flexibility are more time efficient 

In addition, programs that reflect more 
variety in design yield greater time savings 
than those that do not 

Satisfaction 

Another criterion for measurement is 
positive attitudes and satisfaction. The 
majority of studies report positive attitudes 
from trainees for the computer-based train- 
ing system and satisfaction on the part of the 
supervisors or training managers (Kamouri, 
1984). Not all studies were empirically tested 
against traditional methods, however, and, 
therefore did not control for the novelty of 
the training approach. 
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Swissair and British Airways were highly 
satisfied with their new CAI system used in 
training clerks to use new procedures for 
operation (Stammers andPatrick, 1977). Dif- 
ferent organizations and government agen- 
cies have reported satisfaction with CAI 
because of its convenience, money savings, 
and reported favorable employee attitudes 
(Buck, 1982; Leblanc, Bristica, & Evers, 
1978). 

Significantly better attitudes among CAI 
group participants than the traditionally 
trained group participants were reported by 
Schwartz and Long (1967) and Wallis 
(1964). 

In contrast, Wallis (1964) reported that 
trainees engaged in a programmed instruc- 
tion course for electrical mechaniv were 
dissatisfied with the program. After the pro- 
gram was integrated with more variety, more 
favorable attitudes were noted Technical 
trainees at Honeywell using the CAI tutorial 
disliked the inaoility to query the system, 
CRT glare, and slow response time. The 
same participants, however, reported satis- 
faction with the program's thoroughness, 
ability to handle wrong answers, and rapid 
progress through materials. 

In examining the measure of attitudes and 
satisfaction, studies generally showed posi- 
tive results with the introduction of CAI 
packages (Trollip, 1979; Cain, 1981; Con- 
kright, 1982; Weitzman, Fineberg, Gade, and 
Compton,1979). The attitudes of trainees 
toward using the CAI simulators improved, 
after initial issuance, with exposure to the 
system (Weitzc- et al., 1979). 

Negative attitudes toward a flight trainer 
in a CAI program were reported by Crawford 
and Crawford (1984). 



CAI versus Traditional 

Schwartz and Haskell (1966) found no 
difference between CAI and traditional self 
study groups in technical information train- 
ing. 

Another CAI area explored was learner 
performance measures. Wallis (1964) found 
that technicians in an unmodified 
programmed instruction course had lower 
exam grades than those in traditional training 
groups. Schwartz and Haskell (1966) and 
Schwartz and Long (1967) reported achieve- 
ment on technic 1 information post-tests 
comparable for LAI groups and self-study 
groups. Results for centrally integrated 
programmed instruction courses were supe- 
rior to those for basic programmed instruc- 
tion and other traditional methods (Wallis, 
1964). 

Computer-assisted flight trainees showed 
superior performance over traditionally 
trained pilots, attaining criterion perfor- 
mance significantly faster than comparison 
groups (Trollip, 1979; Crawford & Craw- 
ford, 1978) and making fewer critical errors 
(TroUip, 1979). Mitzel (1974) claimed that 
the CAI inservice teachereducation program 
produced better results than traditional 
course performance. CAI course exams were 
24 percent higher than e .:ams for classroom 
courses. 



Summary 



In conclusion, Kamouri (bo4) maintains 
that computers have had an important overall 
affect on training programs. Because they are 
meeting many training needs, industries are 
increasingly adapting computer s>wtems to 
train their employees. It is evident from the 
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research that knowledge acquisition is being 
facilitated The generalizafaiiity of the train- 
ing can be seen in various computer-assisted 
training programs. The use of simulators and 
supplemented CAI techniques seem to be the 
most favorable programs with respect to their 
design and effectiveness in promoting learn- 
ing. These systems incorporate more 
flexibility into the human-computer interac- 
tion. 

The effects of multiple modes or instruc- 
tional supplements, more user control, per- 
sonalized feedback, and increased adaptivity 
to user characteristics were favorable to 
developing skill proficiency, and in some 
cases, training generalizability. 

However, there is a lack of empirical re- 
search on the independent effects of these 
program components and objective perfor- 
mance measures in comparison to traditional 
training methods (Kamouri, 1984). 



Respondent Comments 



The panel members provided information 
about how CAI packages are used in various 
JTPA programs, how well computer-assisted 
instruction serves program objectives, and 
teacher/instructor qualifications. Cited 
programs varied in CAI package utilization. 
Some programs had extensive computer 
centers with 10-15 computers available for 
client use while others were limited to fewer 
than five computers. Most of the programs 
utilize "stand-alone" systems because of the 
expense of networks. The majority of net- 
work systems are housed in programs that 
have purchased U. S. Basics or Control Data 
services. 

The following are respondent comments 
about their experiences with CAI packages: 



In the Adult Learning Center, Control Data's 
PLATO (implemented on a local area net- 
work) is used to provide Basic Skills Reading 
and Math remediation and GED preparation. 
The scope and sequence of the PLATO cur- 
riculum meets the objectives of the Adult 
Learning Center in a most favorable manner. 
In the past year, 76 percent of the Center's 
students met their employability development 
objectives. The PLATO lab instructors arc 
certified teachers who have received ap- 
proximately 5 days of training by Control Data 
before operating the lab. [The lab] has been 
operating for 4 years and the instructors are 
considered national experts on the Local 
PLATO Delivery System. Two out of three of 
the PLATO Lab instructors had no previous 
computer experience. 

CAI packages are currently being used for drill 
and practice and instruction. We feel that they 
do a fair job in serving program objectives. 
Our instructors would be rated as fairly well- 
qualified to use computer-assisted instruction- 
al packages. 

CAI packages are used as an additional tool to 
supplement training. They serve program ob- 
jectives quite well. They seem to be very ef- 
fective. Our instructors seem well-qualified in 
the use of CAI. 

We use CAI for as much instruction and iMii 
as possible. They are being used to review ana 
give practice in all areas of instruction. They 
serve the objectives very well; students who 
would not do flashcards will do the drills on 
the computer. Our JTPA instructors are 
qualified teachers ai; i are very comfortable 
with all of our software. They have received 
computer and software training. They are al- 
ways eager to learn something new. 

CAI packages are specifically selected to meet 
skills identified by the state department of 
education in math and reading. They serve 
program objectives extremely welL Our stu- 
dents show gains in math and reading of two 
to four normal curve equivalent scores 
(NCEs), which is very good. Teachers in our 
lab are certified public school teachers, who 
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may not have had computer experience. They 
participate in a week long training session. 

CAI is used as supplemental instruction with 
students spending 33 percent of their time on 
the computer. The computer instruction is 
primarily drill and practice. We feel that CAI 
serves our program objectives very well Our 
students like working with computers and they 
find their learning more exciting. Our instruc- 
tors do not have to be certified teachers, al- 
though an education background helps. What 
we have found to be more important than 
education is a willingness to learn new things. 

Our clients spend 40 percent of their time on 
computer instruction. We use an assortment of 
software and a computer management sy stem. 
We have found that CAI serves our objectives 
very well. We do not feel our teachers need to 
be computer experts. 

CAI packages are used concurrently with 
other instructional methods. A particular 
package is selected on the basis of strengthen- 
ing a diagnosed reading weakness as deter- 
mined by reading specialist and students 
together. A program objective is to make, the 
program participants independent in the pur- 
suits of goals. CAI helps them do this. 

Interactive CAI packages with an auditory 
component are excellent with ESL clients be- 
cause the computer provides an added feature 
of patience. 

Networked learning systems are more useful 
and effective than stand-alone systems. They 
remove the need to work with hundreds of 
diskettes. A learning system that does not 
operate as a franchise allowr the user to adapt 
the system to local needs and priorities. 

We are moving from stand-alones to net- 
worked systems which are easier to handle. 
Training for stand-alone systems required five 
days at the outset, with three to six days fol- 
low-up per year. Much of that was tied to an 
awkward, complex record-keeping system 
which was required under a franchise opera- 
tion. Staff had to be more computer literate for 
that system and were often bogged down with 



malfunctioning system software and lack of 
knowledgeable technical staff at the corporate 
headquarters. 

CAI is used for instruction with our clients 
spending 50 percent of their time in computer 
instruction. We see CAI as the wave of the 
future for the at-risk population. It is iess 
threatening, more comfortable for the client to 
use, and the client gets immediate rewards. We 
like CAI a lot Our teachers are used as 
coaches and do no classroom instruction. 

CAI is used for remediation in math and read- 
ing, as weii as GED preparation. Computers 
are not going to solve all problems and should 
be viewed as only an aid to the teacher. 
Teachers do not have to be computer experts, 
but they do need to be able to react to students. 



JTPA Program Site Visits 



Purpose 

In order to obtain key information about 
the actual use of CAI packages in JTPA 
programs for the remediation of basic slrilb, 
site visits were made to five selected JTPA 
programs in Pennsylvania and New Jersey. 
A sixth visit was made to a software 
developer. Specifically, qualitative informa- 
tion was sought on characteristics of par- 
ticipants, objectives of the programs, current 
CAI package usage, cost-effectiveness, and 
how well CAI packages serve program ob- 
jectives. Further, JTPA program staff were 
asked to critique the CAI taxonomy and 
proposed criteria for CAI package evaluation 
developed by this project The qualitative 
information gained from :he site visits sup- 
plemented the information obtained from the 
literature. The visits provided more depth 
and detail to an understanding of the variety 
of current CAI usage and provided specific 
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information that should influence the 
development of designs for evaluating the 
effect of CAI packages. 

Format of Site Visits 

Structured interviews were conducted 
with the person in charge of the instructional 
components of the JTPA programs, using the 
same survey questionnaire distributed to the 
expert panelists and consultants. 

The five sites, selected with assistance 
from SD A representatives listed ir the JTPA 
Service Delivery Area Directory, 1988-89 
Edition, were chosen on the basis of geog- 
raphy, demographics, type of CAI packages 
in use, and type of computer system used. 
The programs varied from "high tech" to 
what one instructor called "no tech." One of 
the "high tech" programs was chosen be- 
cause it used a complete CAI curriculum, 
including a management system operating on 
a sophisticated local area network. The staff 
there was highly skilled in use of computers. 
Other sites reflected varying degrees of com- 
petency with computer and CAI usage. 

Contact with site personnel to complete 
arrangements for the site visits was made by 
telephone. The project purpose, question- 
naire, and taxonomy and rationale were dis- 
cussed with the site staff. 

Site Visit Results 

A synopsis of the data collected from the 
site visits are noted below with a description 
of each program. 



Site Visit One 

The site, located in a large urban area in 
one of the poorer sections of the city, serves 
primarily Black clients with multiple needs, 
including low self-esteem and problems as- 
sociated with single-parent families and ex- 
posure to drugs and alcohol. The community 
center housing the program offers a wide 
variety of programs for all age groups. En- 
richment and remedial programs are offered 
for school age children and ABE programs 
for adults. The center participates in a state- 
run experimental program for teen-age 
mothers. The center has used CAI for at least 
a year. 

Private corporations in the city provide 
most of the funding for the various programs. 
A small portion is provided through JTPA, 
Title II. The corporations view support as an 
investment in the development and training 
of prospective employees. 

Private donations paid for the installation 
costs of a Control Data Local PLATO 
Delivery System with 15 PLATO worksta- 
tions and 6 stand-alone Apple He's. The 
hardware for the PLATO system cost ap- 
proximately $100,000. 

Teachers and other center staff par- 
ticipated in the Control Data training pro- 
gram as part of the installation package. The 
center pays $5,000 a year for a maintenance 
contract. Ths center director and assistant are 
knowledgeable about computers and have 
strong backgrounds in math and science. It is 
noteworthy that the staff was very enthusias- 
tic about the PLATO program and comfort- 
able with the technology. They viewed 
computers as a way to better meet the needs 
of their clients. 

Local companies have been impressed 
with the academic progress of students par- 
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ticipating in the center programs. Student 
progress has been documented through the 
computer management system. The system 
documents the progress of participants and 
quantifies the results. The center director 
maintains that the documentation has helped 
in getting more private funding because the 
results were objectively measured. 

The director is adept at operating Control 
Data's Curriculum Management Delivery 
System and is able to program individualized 
curricula for students and to track their 
progress. The stand- alone Apple He com- 
puters, used for supplemental work with 
school children and word processing for 
adults, are not used for instruction because of 
the excessive time required for monitoring, 
given the number of participants. 

The center focuses on enrichment ac- 
tivities for a core group of school age 
children. For adults, the focus is on remedia- 
tion of basic skills and acquisition of job 
skills. For teenage mothers the concern is 
breaking the welfare cycle by providing 
academic and social support. Leadership and 
moral development are a key part of the 
program for all age groups. 

The following are important observations 
made at the site: 

• No matter how sophisticated a computer 
system is, teachers still remain a critical 
component of the instructional program. 

• Orientation sections within CAI pack- 
ages are essential in assisting clients with 
no prior computer exposure through the 
process in a non-threatening manner. 

• All Adult Basic Education CAI packages 
should have feedback systems which 
allow for interaction between the instruc- 
tor and peers. Although computers may 



accommodate the skill and academic 
needs of participants, they cannot deal 
with personal needs. Supplemental ac- 
tivities providing constant reinforcement 
and reassurance and acknowledging suc- 
cesses are needed. 

• The center instructors did not identify 
any significant differences in the learning 
needs and styles of different clients. 
Youth and adults did not differ greatly 
except by age. Displaced workers, accus- 
tomed to a standard of living which they 
try to maintain with fewer resources, tend 
to experience more frustration. They are 
also more prone to quit before complet- 
ing programs if a financial opportunity, 
even if short-term, arose. 

• Because the expense of computer 
hardware is cost-effective only when 
used constantly, the center maintains 
daytime and evening hours six days per 
week and services, without cost, a nearby 
elementary school. 

Site Visit Two 

The site, located in a medium sized city 
with average unemployment problems, is a 
comprehensive JTPA program with basic 
education, job search, summer youth, and 
occupational skills programs. In operation 
for more than 10 years, the program is staffed 
by three instructors, two of whom are new. 
The director has been with the program from 
its beginning and has basic computer 
knowledge. The two teachers are inex- 
perienced with computer usage. 

The ABE program has used CAI pack- 
ages to teach remedial skills and prepare 
students for the GED examination for more 
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than a year. Initial funding for a variety of 
computer equipment and CAI packages 
came from various sources. IBM's PALS 
program is used to teach reading and operates 
on four IBM stand-alone computers. The 
total cost was $75,000. Apple He's use 
"patchwork" software. A Control Data com- 
puter is not used because of limited software 
and lack of instructor skills. 

A major problem has been finding affor- 
dable software to meet the needs of the adult 
population. Several purchases of expensive 
software were unsatisfactory; either the 
software was too difficult for the staff to 
operate or it did not perform as expected. The 
center now previews material before pur- 
chasing. 

The clients served have diverse training 
objectives. A typical day may require ESL, 
basic skills instruction, and preparation for 
the GED exams. Needs within categories 
differ also. For instance, some ESL clients 
may need only language instruction while 
others also need basic skills instructions. 

The program director offered several in- 
teresting observations: 

• Much of the ABE software is not ade- 
quate for instruction because it provides 
mainly drill and practice. 

• CAI should be used in combination with 
other instructional approaches. 

• Both hardware and software must be 
made easier to use and more functional 
a? instructional tools. Many instructors 
arc not adequately trained to use com- 
puters; some have negative attitudes 
against computers as instructional tools. 
Currently, utilizing the computers is dif- 
ficult because of low instructor computer 
illiteracy. 



Site Visit Three 

Located in center-city, the site's JTPA 
funds are channeled through an intermediate 
unit (an administrative liaison between the 
state and local education agencies). Clients, 
aged 14 to 86, vary in characteristics and 
instructional needs. Clients often have low 
self-esteem, are young single-parents with 
several children with no transportation, 
driver's license, telephone, nor marketable 
skills. Many have learning disabilities and 
have experienced little or no success in tradi- 
tional school settings. Teachers in the pro- 
gram are skilled in providing positive 
learning (experiences for such clients. 

An enthusiastic staff has used CAI pack- 
ages for at least a year and a half. The adult 
program purchased Apple He computers 
with cany-over funds from a previous year. 
The teachers are competent users of com- 
puters and software and develop some of 
their own CAI materials. The staff estimates 
that students use computers four to five hours 
per person per week, some to a greater extent 
than others. 

Because of limited resources, purchases 
are carefully considered. Despite these 
precautions, some CAI packages have 
proven not to meet their expectations. For 
example, an ESL program with a manage- 
ment system records student scores in a man- 
ner that has no practical application. 

Pertinent comments and observations 
from the program staff follow: 

• CAI packages should be easy to load, 
easy to access, incorporate a manage- 
ment system, and present materials in 
small increments. 
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• Content of CAI packages should be ap- 
propriate for the age and experience 
levels of the targeted learner. The 
material should be neither offensive nor 
condescending to the learner. Youth do 
not identify with materials which present 
life situations from the perspective of 
middle class, successful adults. 

• Some CAI packages advertised for adult 
learners appear as mere "facelifts," new 
titles covering existing child-oriented 
content. 

• Computer instruction is useful in dealing 
with a vide variety of clients who are 
functioning at different skill levels. Stu- 
dents receive immediate feedback from 
the computer and cannot advance 
without mastery of the material. 

• Software, claiming editing features, 
should contain clear, concise instructions 
for organizing and tailoring lessons for 
the specific needs of individual learners. 

Site Visit Four 

This program, located in a small college 
town, provides services to dropouts and 
adults. JTPA funds are channeled through an 
intermediate unit. Program objectives are 
GED training and remediation of basic skills 
in preparation for other training programs. 
The small program is operated by one in- 
structor. The teacher finds the mixed age 
groups of students beneficial for interaction. 
The older students help motivate the younger 
students. 

One stand-alone Apple He is available for 
participant use with a limited amount of 
software. CAI plays a very minor role; time 
is not scheduled for student usage. The 



primary purpose is exposure to the basic 
routines of loading software, using menus, 
and printing documents. 

The teacher has limited computer 
knowledge and indicated little interest in 
developing further computer skills. A former 
biology teacher in public schools, she is ac- 
customed to teaching with limited equipment 
and instructional aids and does not feel in- 
hibited by a shortage of equipment. She in- 
dicates a strong preference for "traditional 
paper and pencil" method of instruction and 
identifies her program as "no tech." 

Site Visit Five 

This ABE/GED program is housed in a 
vocational school and receives JTPA, Title II 
funding from the local private industry coun- 
cil (PIC). The funds are supplemental, suffi- 
cient only for purchase of materials. The 
primary funding source is the Department of 
Education. The SDA makes referrals to the 
program and may provide the participants 
some financial assistance. 

Clients, aged 16 to 64, are primarily 
female. All meet JTPA income criteria and 
have relatively low incomes. Academic 
abilities vary. Some are high school 
graduates; others have not been successful in 
school. Self-esteem tends to be low. The 
client population is mobile. Attendance is 
sporadic. 

The objectives of the program depend 
upon the needs of the clients. Some receive 
GED training. Others enroll to improve basic 
skills in preparation for college, business 
school, or better jobs. A special program for 
displaced homemakers is provided through 
the intermediate unit. There axe no time 
restraints; a participant may take one week 
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or 16 weeks to successfully complete 
preparation for the GED exam. 

The teacher functions as a counselor as 
well as an instructor, often referring clients 
to agencies which can help them with 
specific problems. She has formal training in 
providing computer instruction. Two Apple 
He* s are used primarily for drill and practice, 
which the staff strongly advocates for student 
achievement The program also has access to 
the vocational school's computer center 
during the summer. 

The staff has developed a software 
evaluation form. Each CAI package is 
previewed and evaluated before purchase; 
problems with purchases are rare. 

Site Visit Summary 

The five site visits were useful in supply- 
ing practical information necessary for the 
development of the CAI evaluation designs. 
Program personnel responded candidly to 
questions. Each of the programs provided a 
different perspective of computer-assisted 
instruction. The following is a synopsis of 
commonalities and pertinent points gleaned 
from the visits: 

• Program staff are concerned with the 
necessity for ease of software utilization 
and hardware operation because of the 
demands upon their time. Packages that 
are not "user friendly" are ignored. 

• The level of computer literacy and con- 
fidence among the personnel varies 
greatly. The majority with major respon- 
sibility for CAI implementation have 
minimal training. Some are apprehen- 
sive; others are enthusiastic about the 
possibilities offered by the technology. 



• None of the programs uses CAI ex- 
clusively. Teachers are acknowledged as 
essential components of the instructional 
process, an opinion affirmed by feedback 
from the expert panelists. This is not 
consistent, however, with some of the 
research on CAI that minimizes the value 
of interaction between teacher and stu- 
dent. The nature of the JTPA population 
mandates positive reinforcement and 
guidance to ensure success during all 
phases of the instructional process. 

• Program personnel do not perceive a 
wide disparity in learning styles and 
needs of clients among the Title n, HI, 
and IV programs. Most youth have life 
experiences similar to adults. Both 
groups have learning difficulties exacer- 
bated by multiple personal problems. 

• Program personnel could not discuss 
CAI cost-effectiveness using concrete 
measures. 

• Presentation of CAI content must be age- 
appropriate in order to accommodate the 
special needs of the JTPA population. 
Instructors indicate a preference for 
materials which allow them to customize 
lesson sequences and content for in- 
dividual learners. CAI packages 
designed for skill development without 
considering the process of learning are 
considered to be of little worth, 

• Most JTPA programs have scarce resour- 
ces. CAI packages are expected to per- 
form as advertised; otherwise, the choice 
will be conventional materials that are 
readily accessible, predictable, andeasily 
assessable. 
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An Assessment of the Cost-Effectiveness of CAI 

Packages 



Levin and Woo's (1981) study of the cost- 
effectiveness of drill and practice packages 
by the Computer Curriculum Corporation 
(CCC) is considered the first systematic cost 
examination of CAI. An analysis of CAI 
cost-effectiveness, Levin and Meister (1986) 
proposed, should include: description of the 
number of students, the duration of the in- 
struction, the necessary personnel, facilities, 
hardware, software, training, and main- 
tenance. A detailed inventory must be com- 
pleted on each of these areas and a cost 
determined. CAI hardware and facilities 
must be allocated over their respective 
lifetimes. 



CAI versus Traditional Methods 



Cost comparisons between CAI and con- 
ventional instruction have been seldom 
studied. Buck (1982) reported a $70,000 
savings with a system having a seven-year 
life-cycle. Because of initial high start-up 
costs, an important factor for cost savings 
analysis, Kamouri (1984) cautioned against 
studies that are not longitudinal. 

Levin and Meister (1986) contrasted CAI 
packages with cross-age tutoring and 
schemes for varying instructional time and 
reducing class size. Their results showed that 
CAI packages were more cost-effective than 
adult tutoring, reducing class size, or altering 
instructional time. Nevertheless, peer tutor- 
ing in mathematics was considerably more 
effective than CAI. CAI was slightly less 
effective than peer tutoring in reading. 



These investigators concluded that while 
CAI may be relatively cost-effective, it is not 
the most cost-effective method in improving 
reading and math achievemen t in elementary 
student*. Levin and Meister cautioned 
against unqualified acceptance of CAI as the 
most cost-effective method of instruction 
and recommended that decision-makers con- 
sider instructional goals, available resources, 
and staff proficiency before choosing to im- 
plement a CAI package. 

A study by Neimiec, Blackweil and Wal- 
berg (1986) disputed the results of Meister 
and Levin (1986), contending that Levin and 
Meister overestimated the achievement gains 
of peer tutoring arid underestimated those of 
CAI. Their study found CAI to be twice as 
effective as peer tutoring. 

These conflicting results indicate the dif- 
ficulty in estimating the cost-effectiveness of 
CAI. Non-tangible factors such as client 
motivation and speed of instruction need to 
be considered. Obviously, further research 
on cost and performance is needed. 



Cost-Effectiveness Analyses 



The literature on cost-effectiveness is 
sometimes unclear because of ambiguous 
terminology. For example, studies inves- 
tigating "training" may use variables related 
to either (a) skills training and instructing of 
students with CAI packages, e.g. ? remedia- 
tion, or (b) the training of instructors to use 
CAI packages with students, e.g., teacher 
inservice. Complicating the assessment 
process is the nesting of (a) costs related to 
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implementation within (b) costs related to 
instruction* The following outline is 
presented to clarify the relationship between 
factors germane to a CAI cost-effectiveness 
analysis: 

INSTRUCTION COSTS 

1. Learning time 

2. Staff size 

3. Materials 

4. Implementation and utilization costs 

a* Installation 

b. Maintenance 

c. Hardware 
d Software 

e. Instructor training 

( 1) Time away from job 

(2) Travel expenses 

f . Trainer 

(1) Time 

(2) Travel expenses 

Conceptualizing this hierarchy will aid in 
deciphering the research results which fol- 
low. 



instruction Costs 



Learning Time 

Time savings studies report significant 
decreases. Reductions by two-thirds have 
been reported for "Criteria Referenced In- 
struction*' pilot training (Kamouri, 1984) and 
teacher retraining (Mitzel, 1974). 



Materials 

Supplemented CAI 

Supplemented CAI systems, those adding 
features like animation, interactive audio, 
touch screens, light pens, mouses, and ad- 
junct workbooks and video, have yielded 
savings in cost (Kamouri, 1984), training 
time (Mitzel, 1974), and instructional staff 
size (Cain, 1981). 

Hardware Costs 

Levin and Meister (1986) estimated ths 
costs of the CCC program, including 10 
minutes of daily drill and practice for stu- 
dents, at $136 per student in 1978. In 1984, 
the costs were re-estimated based on current 
costs, and the program was estimated at $120 
per student. A considerable decrease in the 
cost of computer hardware accounted for the 
decrease in overall costs for the program. It 
was estimated that hardware costs decreased 
from 1/4 to 1/9 of the overall cost of CAI 
programs between the cost estimates made in 
1978 and 1984. Levin and Meister (1986) 
suggest that almost 90 percent of the cost for 
CAI programs is associated with personnel, 
software, training, and other costs. This 
means that further reductions in hardware 
costs can only affect a small percentage of 
overall CAI costs. According to their 1984 
estimates, 38 percent of the total costs were 
for personnel and 21 percent were for cur- 
riculum and software. 

Software Costs 



The numerous types of CAI packages 
available have created a competitive market 
for more than 100 vendors. Currently, for 
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general education, there exists a wide variety 
of software from which to choose. 

Pressman and Rosenbloom (1984) cited 
cost among several factors hampering the 
development of CAI packages. CAI pack- 
ages are designed for different purposes and 
allow for differing amounts of learner inter- 
action. For example, Frenzel (1980) sug- 
gests: 

1. Testing — the computer presents 
questions and grades the students 
answers 

2. Drill and Practice — reinforces 
previous learning by presenting 
practice problems 

3. Tutorial — programmed instruc- 
tion which permits learning in se- 
quential frames with branching to 
permit individualized pacing for 
each student's needs 

4. Dialogue — permits an unstruc- 
tured "conversation" between the 
student and the computer 

5. Simulation — the computer acts as 
an environment, permitting the stu- 
dent to manipulate parameters and 
observe the outcome. 

In addition to considering the cost of 
developing software for various purposes, 
analysis can be conducted along the dimen- 
sion of cost-effectiveness of CAI versus con- 
ventional materials for each purpose, e.g., 
comparing the cost of testing with CAI ver- 
sus testing with pencil and paper. 

Levin and Meister's CAI cost-effective- 
ness study (1986) comparing CAI packages 



with peer tutoring indicated reduced costs of 
CAI packages and their increased perfor- 
mance outcomes as primary factors driving 
the use of computers in instruction in 
schools. The investigators noted that the cost 
of microcomputers had decreased by 50 per- 
cent since 1980 with concurrent growth in 
the variety of software available. 



Implementation and Utilization 
Costs 



Pressman and Rosenbloom (1984) 
analyzed implementation of CAI packages 
by investigating hardware, software, user 
training, installation, and maintenance. A 
recent study by TURNKEY (1988) cited cost 
and lack of staff expertise as two major bar- 
riers to the implementation of CAI packages. 



Installation and Maintenance 

Usually, the installation costs for CAI 
packages and microcomputers are minimal, 
because most packages are installed by 
project staff (Pressman & Rosenbloom, 
1984). In general, vendors have provided 
effective installation instructions and techni- 
cal assistance. Peripherals and accessories, 
including printers, plotters, disk drives, and 
monitors are important cost elements. Addi- 
tional costs may be incurred if extra electrical 
outlets are required. Costs for appropriate 
furniture is another consideration. 

Hardware maintenance costs have 
decreased as microcomputers have become 
more standardized and reliable. The cost of a 
hardware maintenance contract is usually 15 
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to 25 percent of the unit price annually 
(Pressman & Rosenbloom, 1984). 

Software maintenance and other costs are 
complicated by unauthorized use and 
duplication. Because of piracy problems, 
software companies try to recoup their costs 
up front (Pressman & Rosenbloom, 1984). 
Users need to be aware of the licenses restric- 
tions that apply to their use of the software. 
Yearly costs of software maintenance have 
been estimated at 10 to 30 percent of original 
price of the software (Pressman & 
Rosenbloom, 1984). 

Hardware/Software Compatibility 

Over the years, there has been remarkable 
improvement in the availability, variety, and 
quality of educational software. Purchasers 
must consider the problem of compatibility 
of the software with the hardware. 

Training Costs 

Training costs are an essential element in 
an analysis of the cost-effectiveness of CAI 
packages. This is especially true for small, 
rural JTPA programs which lack the resour- 
ces that larger, more urban programs may 
have. In small, rural programs, limited staff 
knowledge and expertise restricts the use of 
CAI packages. Pressman and Rosenbloom 
(1984) saw this as a crucial issue. They were 
concerned with effective CAI training. 
Training costs are high. Manufacturers pro- 
vide a minimum of support, often at extra 
cost to the consumer (Frenzel, 1981). 

Consumers must consider both time and 
money in determining costs. Difficulty of 
learning CAI packages varies with the ven- 
dors and the packages. Often, vendors agree 



to train one or two users. Training of others 
are at extra costs (Pressman & Rosenbloom, 
1984). Large organizations often develop 
their own training programs to help defray 
training costs. This may not be an option for 
small programs. The Minnesota Educational 
Computing Consortium (MECC) was 
developed in the public sector in answer to 
the need for computer training (Pressman & 
Rosenbloom, 1984). 

Training costs should decrease in the 
future due to greater microcomputer use, in- 
creased user expertise, and better CAI pack- 
ages. Pressman and Rosenbloom (1984) 
noted that manufacturers have been assum- 
ing more of the training costs as competition 
has increased. 



Respondents Comments 



The expert panelists* responses were 
generally quite subjective. Most did not have 
detailed information on the actual costs of 
computer-assistedinstruction. The following 
are notable responses: 

The Adult Learning Center is delivering CAI 
for approximately $225 per hour via the Local 
PLATO Delivery System (LPDS). 

Cost-effectiveness depends much on how fast 
hardware changes. 

CAI is very cost-effective once equipment is 
in place. 

CAI packages are extremely cost-effective. A 
$50 piece of software wUl replace several 
books, packs of paper, and tutor assistance. 
Over the one and a half years we have used 
these programs, they have been worth twice 
what we paid for them. 

They are very cost-effective. It cost $150,000 
to start the computer center, inclining 
software and computers. 
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The initial start-up of a CAI laboratory is 
expensive. However, with extended use, the 
number of CAI packages make the program 
cost-efficient. This may not be realized until 
three or four years of laboratory operation. 



Monetary issues play an important role in 
software selection. 

We have made no study on this. However, so 
much enthusiasm and interest are evident 
among youth for computer instruction that the 
investment is always considered worthwhile. 
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Chapter 2 

Criteria for the Evaluation of 
Computer-Assisted 
Instructional Packages 



There are many variables that must be 
considered in the development of the evalua- 
tion designs to assess the effectiveness of 
CAI packages for the remediation of basic 
skills in JTPA programs. One of the most 
important sets of variables that needs to be 
considered in the evaluation concerns the 
CAI package itself. Other related variables to 
be considered in the evaluation design are the 
level of computer skill and training of the 
instructors implementing the CAI package 
and the instructors' level and quality of 
teaching skills. These variables will be dis- 
cussed in detail in Chapter 3 on evaluation 
designs. 

The evaluation design must include 
uniform and objective measures of what CAI 
means in operational terms as used in actual 
JTPA programs. These measures must ac- 



count for the variance in the types of CAI 
packages available and used in JTPA 
programs and for variance in the quality of 
those CAI packages in terms of features re- 
quired to meet the specific needs of the target 
JTPA population. There are many different 
kinds of CAI packages in current use in JTPA 
programs. The manner in which a given 
package is used varies from program to pro- 
gram, and the capabilities of the hardware on 
which a given package is used also varies. 

The various CAI packages vary widely on 
curriculum and software design features and 
on implementation options available. The set 
of variables relating to the CAI packages 
used in the evaluation design should reflect 
the extent to which the packages include 
certain critical curriculum and software 
design features considered most important to 
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the target population. These critical features 
arc those that have been shown to be most 
important in the c tirrem research literature, in 
the findings of the site visits, in the recom- 



mendations of the expert panel members and 
service providers, and in the review and ex- 
amination of various CAI packages provided 
to the design project by vendors. 



Taxonomy of CAI Package Features 



The taxonomy developed by this project 
is a system of classifying CAI packages 
based on the needs of the JTPA population 
and the degree of complexity and quality of 
the technology employed in the CAI pack- 
ages used by JTPA programs in the remedia- 
tion of basic skills. The taxonomy requires 
consideration of three categorical factors 
useful in describing CAI packages: 

1.0 Types of CAI Packages 

1.1 Purpose of the Package 

1.2 Context Orientation of the Package 

1.3 Instructional level(s) of the 
Materials 

It aLo requires consideration of three 
major factors affecting quality of CAI pack- 
ages intended to be used in JTPA programs: 

2*0 Quality Measures 

2.1 Curriculum Features 

2.2 Software Design features 

2.3 Implementation Features 



The taxonomy includes a description of 
various aspects of each of these six factors. 
The taxonomy reflects the concern of 
educators and trainers for giving increased 
attention to the curriculum and related learn- 
ing ac tivities provided through the CAI pack- 
ages. The cOTiculum is a major factor in 
stimulating and guiding student learning. It 
is important, therefore, that CAI packages be 
evaluated not only on their technical merits 
in terms of software design and implementa- 
tion options but also on their curriculum 
design. 

Curriculum design refers to the way in 
which curriculum is conceptualized and its 
major components are arranged These major 
components may be categorized as content, 
instructional methods, instructional 
materials, learner experiences, and learner 
activities, ilie conceptualization and arran- 
gement of the curriculum provide the direc- 
tion and guidance needed as the curriculum 
is implemented in a CAI package through the 
development of the software design* The cur- 
riculum design is the basic framework for 
planning the content of the CAI package. 

Every curriculum design is characterized 
by a certain philosophy of learning, teaching, 
and instruction. The philosophical base in 
this instance is distinguished by its emphasis 
on meeting the specific needs of the JTPA 
learners. To be effective, CAI packages must 
be concerned with JTPA learner charac- 
teristics and learning styles and concerned 
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with the characteristics, goals, and objectives 
of the JTPA programs. 

The following sections discuss two 
groups of major factors included in the 
taxonomy: types of CAI packages and 
quality measures. Within each of these sec- 
tions, three major factors are presented and 
discussed citing specific aspects of the fac- 
tors that have been shown to be important in 
the literature and current research findings. 



1.0 Types of CAI Packages 



CAI packages used in JTPA programs for 
remediation of basic skills arc used by in- 
dividual learners in training programs that 
vary in terms of size and resources available. 
The objectives and r ies of operation cf the 
programs vary. The JTPA participants vary 
in age, experience, and motivation. A good 
CAI package design must take these various 
needs into consideration and address specific 
needs or groups of needs at a specific instruc- 
tional level. The following factors should be 
considered in classifying CAI packages 
based on needs of the JTPA population. The 
rationale for each factor is noted and the key 
aspects of the factors are discussed or high- 
lighted 

LI Purpose of the Package 

A CAI package can be classified accord- 
ing to die purpose or function for which it 
was developed and intended to be used* The 
purposes are presented in the order of com- 
plexity required in the software package to 
accomplish the purpose. A package that is 
primarily intended for drill and pr" ,tice is 
less complex than one that is capable of 
simulating the application of a concept in the 



context of the work place. CAI packages may 
be designed to address a single purpose, or 
they may consist of multiple components 
with different purposes for each component 
In the case of packages with multiple com- 
ponents with different purposes, the in- 
dividual components should be evaluated 
separately in the same manner as a single 
purpose package. The purpose of the CAI 
package can be: 

• Drill and Practice Exercises 

• Supplemental Instruction 

• Primary Instruction 

• Simulation and Application 

Drill and Practice 

These CAI package* are designed to be 
used by the learner for extended practice in 
developing skills, speed, or comprehension 
of previously presented concepts. These 
packages do not teach any new material. 
They are designed to help the learner to 
develop specific learned skills through 
guided repetition and experience using the 
concept 

Supplemental Instruction 

These CAI packages are used by the 
learner as supplemental material to enhance 
and support instruction provided by a 
teacher. The intent is to provide the learner 
with adiitional individual instructional time 
in a self-paced environment These packages 
perform the task of tutoring the learner on 
concepts that have already been presented by 
an instructor. These packages are not in- 
tended to be sufficient for direct primary 
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instruction, but must contain explanatory 
material that the learner can review and study 
to strengthen the understanding and com- 
prehension of previously presented concepts. 

Primary Instruction 

CAI packages which are designed to be 
used by a learner for direct primary instruc- 
tion on specific concepts fill much of the role 
of an instructor* They teach basic informa- 
tion and knowledge about specific concepts. 
The package must be capable of being used 
alone by a learner for sufficient independent 
instruction on the concept The package must 
be designed to introduce and teach the con- 
cepts without requiring other supporting in- 
struction. The package must insure that the 
learner has mastered each critical step in 
learning the concept prior to pro<jnssing 
onto the next step in the process. 

Simulation and Application 

CAI packages which provide simulation 
and application of concepts in a work place 
setting require the learner to use higher level 
conceptual skills of application, analysis, 
synthesis, and evaluation in applying the 
learned concepts to solve work related 
problems. These packages must provide 
realistic workrelated situations with multiple 
possible options available for the learner to 
manipulate in applying the concept The 
learner must receive feedback and informa- 
tion from the software in response to the 
learner's actions anddecisions. The feedback 
should simulate normal behaviors and 
responses that would be expected in the work 
place. 



12 Context Orientation 
of the Package 

The curriculum content of CAI packages 
can be designed and presented to reflect 
various orientations and environments. The 
package can use the functional context of 
work (Sticht, 1987) for instruction or it can 
follow other orientations such as the general 
education and subject matter orientation 
commonly found in CAI packages designed 
primarily for use in public education. The 
context orientation determines the subject 
matter and the nature of examples and situa- 
tions used in the lessons. Within any context, 
the package should teach the critical proces- 
ses important to the remediation of basic 
skills. After learning a basic skill process 
within a context relevant to the learner, the 
learner should be able to perform that basic 
skill in a new context. In other words, the 
learning should transfer to the future contexts 
in which the learners may find themselves. 
The context orientations may be: 

• Specific Functional Context of Work 

• General nctional Context of Work 

• Life Skills Orientation 

• Child Orientation 

Specific and General Functional 
Contexts of Work 

Within the functional context of work, 
only the skills needed for the job and directly 
related to job training are taught All instruc- 
tion in basic skills are related directly to the 
work place and taught in the context of skills 
needeu to perform job related tasks. 
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The functional context for work can be 
highly specific to a particular job — incor- 
porating the technical terms and functions of 
a specific trade or position such as an 
electrician — or it can reflect more general 
work place functions' common to larger 
categories of jobs such as construction 
trades. Many packages use the more general 
work place orientation with the intent of 
being applicable, and therefore marketable, 
to a wider group of training programs and 
clients. Of course, the use of specific func- 
tional contexts of work for instruction are the 
most desirable for training people for those 
specific trades. The general workplace func- 
tional contexts are the next most desirable 
orientations for the curriculum materials. 
Both context orientations are considered ap- 
plications of the concept of functional con- 
text of work. 

Life Skills Context 

Other context orientations of CAI pack- 
ages have no obvious relation to the work 
piace but may reflect a variety of life skills 
and social strings coramon to adults and 
youth in the curriculum materials. These 
packages are df signed for and marketed to 
job training programs, public schools, voca- 
tional-technical schools, and adult education 
programs in which adults and youth are ex- 
pected to participate. The social settings may 
include purchasing groceries, banking, 
paying bills, completing job applications, 
etc. The focus is on functional literacy, func- 
tional skills, and life skills in context of train- 
ing a person to function successfully and 
independently in society as a whole. Th ; ,> 
context may include some general work re- 
lated settings and situations but is rot limited 
to the functional context of the work place. 



Child Orientation 

Some packages were designed for use 
with young children in teaching basic skills 
in elementary schools. The child orientation 
in these packages is intentional and obvious. 
Because of the child orientation, these ph- 
ages may be completely inappropriate for use 
in job training programs with youth and adult 
participants. The orientation will interfere 
with the r urpose of the package by offending 
the participant's sense of maturity and dig- 
nity and by presenting the instructional con- 
cepts in unnatural contexts. 

1.3 Instructional Level(s) 
of the Materials 

The curriculum of a CAI package targets 
particular group of learners at a specific 
instructional level. The entire CAI package 
may be designed for students at a particular 
instructional level, or it may include several 
components that cover a range of instruction- 
al kvels. The instructional level of the CAI 
package must approximate the educational 
achievement level of the learner in order for 
learning to occur. A CAI package with an 
instructional level that exceeds the ability of 
the learner will cause the learner extreme 
frustration and will impede learning and may 
result in damage to the learner's confidence 
and motivation to learn. A package that is 
below the learner's ability may provide a 
temporary sense of accomplishment and a 
review of previously gained skills, but it will 
soon become boring and tedious. An ap- 
propriate level should challenge the learner 
with new concepts and skills presented at a 
level that the learner fully comprehends and 
with tasks that the learner can perform suc- 
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cessfully with moderate effort These in- 
structional levels are classified as: 

• English as a Second Language (ESL) 

• Basic Literacy 

• Adult Basic Education (ABE) 

• General Equivalency Diploma (GED) 

• Advanced Skills 

English as a Second Language 

The instructional level of English as a 
second language CAI programs is difficult to 
classify since knowledge of the English lan- 
guage is not directly related to the learners 
level of educational achievement. A limited 
English speaking learner may be at almost 
any educational level and need English lan- 
guage instruction. Within ESL instruction, 
the instructional levels are reflected with 
such terms as beginning, conversational, in- 
termediate, and advanced language instruc- 
tion. However, the ability of an ESL student 
to benefit from participation in the upper 
levels of ESL instruction will be limited by 
their general educational ability. The upper 
level ESL instruction typically expects the 
student to be literate in their native language 
and to transfer skills in reading and writing 
to English. ESL does not and should not 
include teaching the basic concepts and skills 
of reading. In current practice, ESL pro grams 
follow a transitional model which teaches 
beginning and conversational levels of 
English with limited text oriented materials 
and then transfers the students to 
"mainstream" instruction in English at the 
appropriate academic level. 



Basic Literacy and Adult Basic Education 

The lower levels of basic literacy (grades 
0-4) and adult basic education (ABE) (grades 
5-8) tend to follow the public school grade 
level structure, although the specific grade 
level reference is sometimes replaced with 
some other label for the levels. The basic 
reading concepts and skills including word 
attack, reading comprehension, vocabulary, 
language arts, and writing skills are taught in 
well d fined sequence and increments. The 
basic mathematics concepts and skills are 
taught including units of measure, addition, 
subtraction, multiplication, and division with 
integers, positive and negative numbers, 
fractions, and decimals. 

General Equivalency Diploma (GED) 

The GED level CAI packages are com- 
parable to high scnool level material. Read- 
ing skills and written romposition skills are 
taught and developed including speed, com- 
prehension, vocabulary, logic, and grammar. 
Mathematics concepts, computation, and 
problem solving are taught through the levels 
of basic algebra and geometry. Beyond the 
basic skill areas of concern to JTPA, some 
CAI packages specifically designed for GED 
preparation include the other general educa- 
tion areas included in the GED examination. 

Advanced Skills 

Skills beyond the high school instruction 
level are required by many positions. The 
instructional level is equivalent to college 
and vocational-technical level instruction in 
mathematics, English, and the arts and scien- 
ces. The training of displaced workers for 
new jobs in skilled and technical areas re- 
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quires specialized CAI packages *vhich ex- 
ceed the high school level instruction in cer- 
tain technical aspects of the training. 
Professional development programs also ex- 
ceed the high school instructional level and 
may include highly sophisticated and spe- 
cialized training and simulations. 



2.0 Quality Measures 



The quality measures of the features of a 
CAI package determine the extent to which 
the package is capable of meeting the needs 
of the JTPA population. A defined set of 
features has been determined to be important 
to JTPA students. These features are consis- 
tent with the principles of sound curriculum 
design, software design, and implementation 
options for CAI packages. A few of the 
aspects of these quality features are specific 
to JTPA students and are based on the learn- 
ing styles most commonly found in the target 
population. CAI packages found to possess 
the desired features are expected to be most 
effective in meeting the remedial basic skill 
needs of the JTPA students. 

2.1 Curriculum Design Features 

The most important curriculum design 
features of CAI packages used in JTPA 
programs for the remediation of basic skills 
can be grouped under the categories of con- 
tent, flexibility, feedback, and learning 
styles. The curriculum provided through the 
CAI packages vary in their assumpt ; :ns 
regarding the nature of the educational pro- 
gram in which the package will be used. 
Many JTPA programs are characterized by 
short-term programs in which learners have 



considerable control over the progress of 
their learning. 

On the contrary, some CAI package are 
based on the general public school educa- 
tional model in which the program tends to 
be long-term and under the guidance of a 
teacher. The curriculum is sequential and 
comprehensive, and evaluation is ac- 
complished by periodic testing. It is assumed 
that teachers are available to provide any 
needed additional help in understanding the 
concepts presented. Such packa£ • do not 
contain the features shown f o be important to 
successful CAI in the literature and research. 

The CAI package curriculum must in- 
clude a number of important features in order 
for the curriculum to be complete. The fea- 
tures under each category that are judged to 
indicate a high quality CAI package in terms 
of curriculum design are listed below. 

Content 

The material is competency based and 
teaches the competencies required for ade- 
quate levels of job performance in the func- 
tional context of work. The material is 
presented in an appropriate form to motivate 
the learner, develop confidence in the ability 
to learn, and enhance 'earning under the con- 
trol of the individual learner. 

• Lesson objectives are clearly defined 

• Concepts are covered in depth adequate 
for the purpose of the package 

• Content is consistent with the instruc- 
tional level 

• Curriculum is a mastery based approach 
to job related competencies 



38 



• Content includes testing to document 
mastery of competencies 

• Concepts are reviewed and reinforced 

• Vocabulary, concepts, and examples are 
relevant to learner 

• Lessons are manageable in size 

• Content is logical and well-organized 

• Material reflects current knowledge 

• Content is grammatically error free 

• Content is factually correct 

• Content provides clear and effective 
demonstrations and examples 

• Format is challenging but not frustrating 

• Content avoids ethnic, racial, or sexual 
discrimination and stereotyping 

Flexibility 

The learner and/or instructor may tailor 
the instruction to meet particular job related 
or perceived needs rather than following a 
sequential learning process through a 
prescribed curriculum sequence. Through 
the software design, the learner should be 
able to use menus to make decisions such as 
repeating a series of exercises, reviewing 
instruction, seeking help, checking mastery, 
or exiting the package. 

The content of a special set of items 
should be modifiable so the instructor can 
add and change special items to address 
topics of local or temporary importance. The 
package should provide the framework 
needed for these items. The instructor will be 



expected to provide the text of the instruc- 
tion, the text of questions or exercises, the 
response options, and the correct answer. 

• Difficulty level may be adjusted by the 
instructor and/or learner 

• Some items can be customized by the 
instructor. 

Feedback 

The learner expects immediate feedback 
on performance from the educational ex- 
perience and may avoid periodic testing. The 
learner expects to be able to get additional 
help from within the learning materials to 
clarify learning without relying on others for 
explanations. After a wrong response is 
detected, the package should loop to a new 
presentation of the concept or a help screen 
that provides additional instructional infor- 
mation or clues, and then allows the learner 
to retry the item or a similar item. Multiple 
wrong responses should result in a 
demonstration or example of the correct 
response and return the learner to a point 
where the instructional routine can be 
repeated or the learner can exit the program. 

• Immediate feedback to learner on % *rfor- 
mance 

• Help routines are available for clues and 
examples 

• Multilevel branching after wrong respon- 
ses to help routines and options 

• Tone of address is appropriate to 
learners 

• Feedback is useful and supportive of 
learning 
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• Summary of learner performance is 
provided 

Learning Styles 

CAI packages must meet the diverse 
needs of JTPA participants by being sensi- 
tive to different learning styles. CAI pack- 
ages should encompass the auditory, visual, 
and tactile/kinesthetic approaches to learn- 
ing. The package should include materials 
that address all the approaches. The software 
should include clear instructions to the 
learner on how to use any other materials 
provided- The package should coordinate 
and integrate the use of the various materials 
with the content of the software. 

• Content coordinates and integrates use of 
other materials in the package with the 
software 

• Content includes visual reinforcement of 
learning. 

• Content includes auditory reinforcement 
of learning. 

• Content includes kinesthetic and tactile 
reinforcement of learning 

22 Software Design Features 

There are many aspects of software 
design that are important in the selection and 
evaluation of CAI packages. The software 
design delivers the curriculum content con- 
tained in the software to the learner. It deter- 
mines the nature of the teaching — learning 
environment on the computer. Through the 
software design, the author of the software 
(the instructional designer) plans an interac- 



tion with the learner. The instructional desig- 
ner can incorporate sight, sound, and touch 
in the design to stimulate the learner and 
attempt to elicit planned emotional and intel- 
lectual reactions. Of covxse, the personal 
characteristics and psychological variables 
of individual learners will affect the respon- 
ses obtained from the stimulus. The common 
set of characteristics of the target group of 
learners must be carefully considered in 
designing the software to achieve the greatest 
likelihood of desired responses. 

Software design involves a complex set 
of factors. Each factor contributes to the ef- 
fectiveness of the overall design. 

An effective software design will permit 
the curriculum content to be delivered in a 
manner most appropriate for the target group 
of learners. Appropriate software design will 
allow the learner to obtain the maximum 
learning benefits from the material being 
presented. The amount of effort required of 
the learner to interact with the software will 
be rewarded by a sense of satisfaction. The 
learner's interaction with the software will 
result in immediate delivery of desired, ex- 
pected, and relevant information and 
response from the software. The concentra- 
tion of the learner should be focused on the 
curriculum material and content and not on 
the process required to interact with the 
software. Interaction with the software needs 
to be as natural and intuitive as possible. 
Once a routine of interaction with the 
software is established, it is desirable to fol- 
low that routiro consistendy throughout the 
software package. 

Poorly designed software will interfere 
with the learning process by frequently dis- 
tracting the learner from the curriculum con- 
tent and focusing attention on the process 
required to use the software. Poorly designed 
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software can create frustration and can inter- 
fere with learning even when the curriculum 
content being presented in the software is 
well designed and otherwise effective. 

The following factors are important con- 
siderations in software design. Each factor is 
described and discussed in terms of criteria 
that should be considered in the development 
of the software. 

User Interface with Equipmr nt 

The manner in which the user physical'/ 
interacts with the software is through the 
senses of sight, sound, and touch. There are 
several aspects in each of these modes that 
are important in software design. 

Visual interaction is through a monitor. 
The primary variables that arc important are: 

• Color: selection and range of colors in 
the display is effective 

• Resolution: images- are clear and 
detailed 

• Recognition: images are realistic and 
easily recognized 

• Complexity : a reasonable number of im- 
ages are displayed at one time 

• Image size: physical size of images and 
characters is appropriate 

• Animation: any movement of images 
supports learning 

• Attraction: techniques to focus or se- 
quence attention are effective 

• Speed: time required to generate images 
is reasonable 



Sound can be produced by software and 
reproduced by various types of speakers. 
Recorded sound can be digitized and stored 
on disks. Digitized sound produces a more 
natural and higher quality sound than that 
prouuced by software manipulation of the 
frequency and duration of electrical current 
The problem is that digitized sound uses 
enormous amounts of computer storage 
space. Some software places digitized sound 
on compact disc read-only memory (CD 
ROM) which reproduces the human voice 
but can be written on only once at present 
Mechanical access to data on the CD ROM 
is very slow resulting in tedious delays in the 
programs. The variables important to sound 
and touch are: 

• Sound Quality: sound is realistic 

• Devices: use of headsets and speakers are 
supported 

• Adjustment: learner can adjust tone and 
volume of sound 

• Keyboard: use of keyboard is commen- 
surate with level of instruction 

• Mouse: use of a mouse is supported for 
selection of menu options 

• Lightpen: use of a lightpen is available 
as an alternative to a mouse 

When images and sound are used 
together, the displayed image and the sound 
or voice presented to the learner should be 
coordinated and complementary. The learne** 
will be distracted or confused if the image 
presented and sound heard are unrelated. 

The transmission of responses from the 
learner to the software is typically through 
motions of the hand on some device designed 
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to control a cursor on the screen. The general 
options available are keyboards or number 
pads, light pens, and a mouse or tracking ball 
device. A touchscreen is another option 
available for some hardware. The learner 
makes a menu choice by touching the screen. 
The location of the learners finger on the 
surface of the monitor is detected by a pres- 
sure sensitive grid on the surface of the 
monitor or by interruption of an array of 
lights and light-sensors around the edge of 
the monitor. 

Although computer keyboards retain 
much of the standardization of the typewriter 
keyboard, they generally add numerous other 
keys for cursor control, numeric input, and 
specialized functions. Keyboards, efficient 
time-saving devices to experienced users, 
can be complex ~ 1 intimidating to initial 
users. However, some users report benefit 
from learning to use the keyboard while 
simultaneously remediating basic skills 
when the software is appropriately designed- 
Verbal responses of the learner in the 
interaction with the software can be recorded 
and stored by various devices. The response 
can be digitized and stored on disk for later 
access and xeproduction in high quality or it 
can be stored on magnetic tape using a tape 
recorder. 

Current technology allows only limited 
use of verbal commands from the learner to 
control the software. At present, voice recog- 
nition has not been developed to the point 
where it is a reasonable option for controlling 
educational software. 

Functional Design Components 

The control of the flow of action and 
events in the software allows the learner or 
the instructor to tailor the lessons to the 
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specific needs of the learner. Control allows 
the user to select from available options 
provided by the software and to exercise 
choice in the operation and the duration of 
various routines and lessons. 

Menu Systems 

There are a variety of systems used to 
provide the learner with choices for control- 
ling the software. Choices can be provided in 
a menu displayed on the screen, accessed by 
a mouse, or as functions assigned to specific 
function or control keys or combinations of 
keystrokes which can be invoked with or 
without a menu display. 

Menus may be displayed in the following 
manner: 

• Full screen displays are used effectively 
to provide detailed instructions to the 
learner. 

• A small permanent section, generally a 
bar at the top of the screen, can be ac- 
cessed by a specific keystroke or by 
touching the bar with the cursor using a 
mouse. The small section typically con- 
tains titles of other menus. These menus 
are contained in a temporary dialogue 
box which expands down from the 
selected title either automatically when 
touched by the cursor or by "clicking" a 
mouse. The dialogue boxes contain menu 
options that the learner may select to 
control movement from option to option 
within the software. Options can activate 
a lesson, a help routine, or a check of 
mastery of a concept 

• Similar to a dialogue box, a "pop-up" 
box, may be activated by a specific 
keystroke sequence or a function key. 
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The "pop-up" box appears somewhere on 
the screen and overlays the normal dis- 
play. The box may contain information 
such as a clue, a formula, a reference 
table, a vocabulary list, or additional in- 
structions; or it may contain a utility such 
as an on-screen calculator for student use. 
The "pop-up" box differs from the 
dialogue box in that it provides additional 
information or a utility for student use 
rather than a menu of choices to change 
the flow of the lesson. 

• Software can provide a means for the 
learner to turn menus that occupy a por- 
tion of the screen on or off as desired to 
allow the learner to select to have con- 
stant display of the menu of choices or 
select to have a less cluttered screen on 
which to work. 

It is important that the learner be able to 
reach and activate any menu choice rapidly. 
Systems that require extended effort to cycle 
through long series of unwanted menus in 
search of a desired choice are frustrating and 
time-consuming to use. Software design 
should group related option sets in the same 
dialogue box and allow random av :ess for 
selection and activation of choices within the 
box. 

The software design must Jlow for 
smooth and rapid exit from one menu to 
another with minimum effort. Menus should 
provide a non-destructive means to cancel or 
abort the action selected to allow the learner 
to escape from an unintended choice. 

Design Components 

The design components are routines that 
are provided for use by the learner or the 
instructor supervising the use of the software. 



These components allow the software to be 
used more efficiently with particular 
learners. They provide a means of defining a 
set of options and choices that will adapt the 
software to the needs of the learner. These 
routines include: 

• Diagnostic routines to assess the level of 
performance of the learner in terms of the 
lessons available in the CAI package 
either as part of the software or as other 
material provided in the package 

• Unit planning assistance to suggest and 
develop relevant groups of lessons based 
on learner performance and diagnostic 
routines 

• Flexible lesson sequencing to allow the 
definition of a selected series of activities 
and lessons tailored to the needs and in- 
terests of the learner 

• Multi-level lessons sequences that pro- 
vide relevant units of instruction for 
adults who are functioning at various in- 
structional levels in different content 
areas. 

• Flexibility to easily branch to more dif- 
ficult or easy instructional levels within 
lessons when needed 

• Random use of items from a pool of items 
at a given level of difficulty — designed 
for use in a particular lesson to allow 
repetition of the lesson with a new set and 
sequence of items 

Interaction with User 

The software design should develop and 
maintain a supportive and positive emotional 
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climate in its interaction with the learner. Tne 
software should provide adequate hints and 
explanations to effectively communicate to 
the learner what is expected for satisfactory 
response. It should allow the learner to edit 
work and entries easily. The necessary edit 
procedure may vary from a simple change of 
a single entry on the screen to a full screen 
editing of text 

The software needs to provide immediate 
feedback to the learner on performance and 
allow missed items and concepts to be 
repeated until the learner has gained mastery. 

The software should follow consistent 
routines of interaction with the learner so that 
the learner can anticipate the next event and 
concentrate on the subject matter rather than 
the process of interaction. 

The software should provide quick 
responses to all entries by the learner. The 
learner should not be concerned with pos- 
sible reasons for delays or blank screens. The 
learner should have the ability of halting a 
lesson and returning at a later time. The 
software should expect that learners will 
enter and exit the lesson sequence at varying 
points. 

The ability of the learner to mak/; choices 
and exercise a degree of control over the 
learning process is very important. The 
process encourages independent decision- 
making. It develops and encourages the 
learners to assume personal responsibility for 
learning. The learner needs to have control 
over the repetition of lessons and the pace of 
instruction. The learner and/or the instructor 
need to have control over jumps to more 
difficult or to easier materials and the intro- 
duction of new materials and concepts. The 
learner needs to be able to enter and exit 
programs freely and to return to any point in 
the sequence of lessons. Time available for 



computer-assisted instruction may vary 
greatly from day to day depending on many 
factors. 

Management Services 

Software design should provide the in- 
structor with options for keeping records on 
the learners using the software. These 
records should include reports by individual 
learner and by groups or classes of learners. 
Instructors need to have records maintained 
that show which lessons they selected to be 
included in a learning unit (based on diagnos- 
tic information) for each student. The records 
should be available both as printed output 
and as screen display. The management ser- 
vices should include routines to track in- 
dividual learners in terms of: 

• Lessons included in learning units 

• Lessons assigned 

• Lessons completed 

• Performance on individual lessons 

• Summary performance on learning units 

• Amount of time learners spend on les- 
sons 

• Number of times a learner attempts a 
lessen 

2.3 Imple m entation Fea tures 

The implementation features of the CAI 
package is an important consideration. Im- 
plementation features include the way the 
CAI packages are installed, maintained, and 
operated on microcomputer hardware. 
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Installation 

Installation of the CAI package on project 
hardware can be one of the most frustrating 
stages of using CAI packages. At this stage, 
the project staff may have little or no ex- 
perience with the CAI package. The installa- 
tion process can be complex involving 
several steps requiring decisions about 
monitor type and resolution; locations for 
storage of programs, lessons, and work files; 
printer and communications options; key- 
board, mouse, and lightpen options; memory 
requirements and allocations; local area net- 
work and security requirements; and sound 
options. The process involves copying 
numerous files from the vendor's distribu- 
tion disks to the disks in the project's com- 
puter. The copying can be accomplished by 
installation programs provided by the vendor 
or by detailed instructions to the person in- 
stalling the CAI package. Improper installa- 
tion can result in the CAI package not being 
used at all or the package being partially used 
with less than optimum expected perfor- 
mance. 

CAI vendors generally assume that instal- 
lation and maintenance of the CAI package 
will be accomplished by a project staff mem- 
ber who has a working knowledge of the 
computer hardware used at the project site. 
Although many vendors provide limited 
technical assistance via telephone to assist 
project staff members in the installation 
process, the written materials provided with 
the CAI package provide the main installa- 
tion instructions. The quality of these written 
materials is very important to successful and 
correct installation of the CAI package. 

The installation instructions should make 
no assumptions regarding the hardware. 
Each requirement should be clearly stated. 
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All requirements and any required modifica- 
tions to configuration ana system files should 
be stated explicitly with detailed and com- 
plete examples provided. 

Any hardware or operational limitations 
and known equipment conflicts of the pack- 
age that would affect installation must be 
clearly explained. The software design 
should not make any unnecessary restrictions 
on installation. A common problem involves 
the vendor assuming that the software will 
always be installed in the simple pattern of 
program files being stored on Drive C: of a 
hard disk and work stored on floppy disks in 
Drive A:. If these assumptions are followed 
to the point that drive designations are in- 
cluded in the vendor provided installation 
urograms, it may be impossible to install the 

I package on more complex storage 
schemes commonly in use. 

Each step in the installation process must 
be clearly describedin the written installation 
materials and operate error free. Examples 
should be provided for each command re- 
quired to correctly install the software and 
the responses resulting from the commands. 
The written material should include illustra- 
tions of the screen when several options are 
presented, and each option should be dis- 
cussed in the installation materials. 

Speed of Operation 

Speed of operation of the CAI package is 
an important factor. The package should 
operate smoothly without excessive delays 
for loading data, preparing graphic displays, 
or performing other routines. Speed is af- 
fected by: 

• Program design 
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• Microprocessor 

• Video display 

• Data storage devices 

The programming used in the CAI pack- 
age affects the speed of operation. A quick 
res onse to any input by the learner should 
be expected The software design controls 
the sequence of events. An immediate 
response to learner input can and should be 
built into the software design. The response 
may vary from an immediate execution of the 
response to an immediate change in the dis- 
play that indicates to the learner that the inpur 
has been received and action is being taken. 
If a delay is expected, a message or icon 
should be displayed explaining what is hap- 
pening. 

The speed and power of die microproces- 
sor will have a significant impact on overall 
operational speed. The larger the amounts of 
data that can be handled at one time by the 
microprocessor, the faster the operation. 
Thus, the 32-bit microprocessors are more 
powerful and operate much faster than the 
16-bit and 8-bit microprocessors. Clock 
speed measured in megahertz (MHz) also 
affects speed directly. A 32-bit microproces- 
sor operating at 25 MHz is 56 percent faster 
than a 32-bit microprocessor operating at 16 
MHz. Some complex programs may require 
the use of more powerful and faster 
microprocessors for satisfactory speed of 
operation. Of course, the most powerful and 
fastest microprocessors arc much more ex- 
pensive than the common models. Most CAI 
packages should operate with acceptable 
speed on the less expensive machines. 

The speed of the video display also affects 
overall operational speed. A graphics display 
using pixels to form letters, such as is used 



exclusively in the Macintosh, requires the 
transfer of enormous amounts of information 
from memory to the video screen each time 
the screen is rewritten which results in a 
significant time delay. A display based on 
text, such as is common in IBM compatible 
machines, is much faster but less dramatic. 
All graphic displays, IBM or Apple, require 
frequent transfers of large amounts of video 
data and, therefore, require powerful and fast 
microprocessors for reasonable speed of 
operation for the CAI package. The basic 
relationship is: the greater the resolution and 
complexity of the image being displayed, the 
greater the amount of data that must be trans- 
ferred and displayed, and the greater the time 
required to perform the required tasks. 

Speed of operation is greatly affected by 
data access time. Fast access to data on a hard 
disk is important. The mechanical speed of 
operation of hard disks vary considerably. 
The access time is measured in milliseconds 
(ms). Presently, fast IBM compatible hard 
disk drives access data at about 18 ms com- 
pared to the 65 ms access on hard disk drives 
commonly used in the past. Mechanical ac- 
cess to data on floppy disks is very slow 
compared to access on hard disks. Access to 
large amounts of data on compact disk read- 
only memory devices (CD ROM) also tends 
to be very slow. 

Performance Options 

Error free operation is expected of all 
software. User error handling and recovery 
from user error should be designed to avoid 
sudden exits from the program and provide, 
simple and clear information to the user 
about the nature of the error and what was 
expected. 
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The software should allow the use of a 
variety of peripheral equipment including 
printers, monitors, and disk storage devices. 

Printer options should be available to 
allow the software to function with a wide 
variety of commonly used printers. Installa- 
tion of printer drivers should be flexible so 
printers can be changed easily. 

Cursor control options should include the 
use of a mouse. Other options should be 
clearly specified and explained in the instal- 
lation guide. Monitor opticas and drivers 
should be provided to accommodate all 
major types of monitors on which the 
software can be used. If color or high resolu- 
tion monitors arc required, the requirement 
should be clearly stated in all literature. 

Disk s;rrage options should include the 
expected use of a hard disk system. No as- 
sumptions about drive assignments should be 
made. Options for use of floppy drives 
should be available and anticipated for stu- 
dent work disks. Individualized student work 
disks should be an available option for 
storage of student progress and performance 
data. 

Installation and use of .the software on a 
local area network should be anticipated and 
accommodated. Software should be avail- 
able in both nework and stand-alone ver- 
sions. The commonly used local area 
network (LAN) systems (such as Novell, 



3COM, IBM Token Ring, TOPS, and Apple- 
Talk) pre Adc projects with a cost-effective 
means of managing the use of CAI packages 
on hard disk servers. Less expensive 
workstations can access the mo/e powerful 
central servers. The multiuser, multiproces- 
sor LAN's provide many additional features 
beyond those available on stand-alone 
microcomputers including security systems, 
shared hard disk storage, shared printers and 
print spooling capability, multiuser file and 
record locldng capabilities, and overall sys- 
tem control and monitoring. 

Maintenance and Technical Support 

Maintenance and updates should be 
provided following a reasonable policy for 
registered users. Documentation for the nor- 
mal operation of the CAi package must be 
carefully written, accurate, and comprehen- 
sive. The documentation should include ex- 
amples for all command sequences required 
to operate the software. Toll free (800 num- 
ber) telephone technical support should be 
available for registered users. The vendor 
should provide options for clients to attend 
periodic in-service training sessions con- 
ducted by the vendor or agents of the vendor 
for extended training and technical assis- 
tance. 



Evaluation Form 



An evaluation instrument has been 
designed to establish values for each of the 
factors in the taxonomy. The instrument will 
produce a standardized and objective means 
of categorizing CAI packages and producing 
standardized numeric measures of quality for 



each of the features considered important for 
effective use of CAI packages with the JTPA 
population. The evaluation instrument will 
be used as part of the evaluation designs by 
skilled and trained evaluators employed by 
the Contractor performing the evaluation to 
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examine and rate various CAI packages used 
in JTPA programs. 

The evaluation instrument will yield three 
categorical variables describing the CAI 
packages in terms of: 

• Purpose of the Package 

• Context Orientation of the Package 

• Instructional Level(s) of the materials 

And, the instrument will yield three 
numeric measures of quality, one each for: 

• Curriculum Features 

• Software Design Features 

• Implementation Features. 

These categorical variables and numeric 
scores will be used in the numeric analyses 
of data in the evaluation design. 

This form is intended to be used by 
evaluators who have experience with con- 
cepts of curriculum design and software 
design. The evaluators will be trained to 
categorize the CAI oackages according to 
type and to describe t.:& functional aspects of 
the CAI packages in terms of hardware re- 
quirements and available options. 

The evaluators will undergo training to 
standardize their understanding of the mean- 
ing and intent " *ach of the items on the 
evaluation form. They will be trained to 
apply the form to evaluate actual CAI pack- 
ages to establish numeric measures of quality 
for the curriculum, software, and implemen- 
tation features found in the CAI package 
being evaluated. 

The taxonomy and the evaluation instru- 
ment will serve the important role in the 
evaluation designs of operationalizing, es- 



tablishing values for, several of the important 
variables related to the various CAI packages 
actually used in JTPA programs. They also 
will provide a means of obtaining a uniform, 
objective rating for various CAI packages 
intended for j in JTPA programs that may 
have utility outside of the olanned evaluation 
process. 



Pilot Test of Evaluation Form 



A site visit was made to a small software 
company near the Pennsylvania-Delaware 
border after the taxonomy and evaluation 
form had been revised with feedback -Tom 
exert panelists, consultants, service 
providers, and JTPA site visits. This contact 
was made on the recommendation of an ALT 
consultant who acknowledges the owners as 
experts in the development of CAI packages 
and has installed the company's materials in 
ABE/JTPA programs statewide. The pur- 
pose of the visit was to pilot test the validity 
and utility of the evaluation form on software 
specifically designed for the remediation of 
bar • skills in JTPA programs. The project 
staff wps also interested in the development 
and utilization of CAI packages from the 
vantage point of business and industry. 

The software developers at the BLS Tutor 
Systems site were highly knowledgeable 
about both curriculum content and software 
design. Its staff randomly selected a com- 
pany-designed ABE CAI package for the 
pilot test. Each section and item of the 
evaluation form was analyzed for clarity and 
relevancy. Ambiguous statements were 
marked for modification. Items representing 
philosophical differences among and be- 
tween software designers and educators, e.g., 
learner versus teacher control of content se- 
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quencing and difficulty level, were identified 
so that concomitant ramifications for the 
rating scale could be predicted 

Personnel of Bidwell Training Center, 
Inc., also assessed the utility of the form with 
CAI packages used in their Literacy and ABE 
programs. The final version was tested by 
project staff with the CAI packages provided 
by the participating vendors (Appendix). 
Notes maintained during the five-hour ses- 
sion with BLS personnel were used by the 
project staff to refine and improve the evalua- 
tion form's design and utility. 



Rating Scale 



The form will be used by the evaluators 
to assess a CAI package in relation to the 
quality criteria presented Each CAI package 
will be rated in terms of the extent to which 
*ach of the items contained in the form was 
presentedin the package being reviewed For 
each item, the following scale is used: 

NA Not Applicable to this CAI package 
(A category to be used in rare in- 
stances where an item is not an 
appropriate feature for the pack- 
age.) 

0 = Feature does not exist in the package 

1 = Feature exists but is very poorly 

presented throughout 

2 = Feature is presented but needs 

some improvement 



4 = Feature is presented exceptionally 
well throughout 



Scoring Process 



A scoring process described in this sec- 
tion will produce a numeric score for each of 
the three qu- ,; «y measures: Curriculum 
Design Features, Software Design Features, 
and Implementation Features, The steps in 
the process are listed below: 

1. A trained evaluator will operate the 
CAI package according to the in- 
structions provided with the pack- 
age on the type of computer equip- 
ment on which it was intended to 
operate by the ver Jor. 

2. The trained evaluator will score the 
CAI package on each item in the 
evaluation form using the scale 
described above- For each feature, 
the evaluator will place a check in 
the box under the rating score given 
to the feature. 

3. The evaluator will count the num- 
ber of checks given in each rating 
column for each of the three quality 
measures and will enter the total in 
the boxes provided in the Count 
row at the end of each quality 
measure section. The total number 
of NA (not applicable) ratings will 
also be entered in the appropriate 
box provided 



4. The evaluator will multiply the 
3 = Feature well presented count by the rating value and enter 

the product in the boxes provided 
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in the Total Score row. The sum of 
the boxes in the Total Score row 
will be entered in the box labeled 
Sum in the Total Score row, 

5, The number NA items will be 
subtracted from the total items in- 
dicated in the Final Rating row to 
find the number of Valid Features. 
The Sum in the Total Score row is 
divided by ihe number of Valid 
Features to yield a numeric Rating 
for each of the three quality 
measures. This Rating is entered in 
the box provided The Rating is on 
the same four-point scale used to 
rate each of the individual features 
as described above. The decimal 
place should be rounded to two 
places. 

The rating for each jf the three quality 
measures can be conceptualized as grades for 
the CAI packages using the common four- 
point grade scale: A = 4.00, B = 3.00, C = 
2.00, D = 1.00, and F = 0.00. 



The Ratings provide a numeric scores to 
represent the quality of the CAI package in 
the evaluation design. The three types of CAI 
package items will provide categorical 
measures to be used in the evaluation design. 

Space is provided on the form for note 
codes. Note codes will be used by the trained 
evaluators to refer to standardized com- 
ments, limitations, explanations, and 
qualifications that provide additional infor- 
mation ' .jout the extent to which features are 
available and presented in the CAI packages. 

The CAI Package Evaluation Form 
presented on the following pages is intended 
to be printed on a single page, front and back, 
with two folds so that there are three "pages" 
on each side of the form. The inside three 
pages will contain sections 1.0 and 2.0 (page 
52 of this report), section 2.1 (page 53), and 
section 2.2 (page 54). The outside or back of 
the form will contain the cover page (page 5 1 
of this report), section 2.2 Continued (page 
55) and section 2.3 (page 56). 
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C AI Package Evaluation 

Form 

DescripPnn o f the CAl Package 

Name: Contents of Package: 



Version: 
Publisher: 



CAl Package Code No: 
Subject Code: 
Hardware Requirements: 

Hardware Limitations: 



Date of Review: 
Hardware Used fcr Review 

Reviewers: 
Quality Ratings: 

Curriculum Design Features: 

Software Design Features: 

Implementation Features: 
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OA! Package Evaluation Form 

1.0 TYPES OF CAI PACKAGES 



drill and practice exercises 




sucolementaJ instruction 




primary instruction 




simulation and application 




1.2 Context Orientation of the Packaae 




specific functional context of work 




qenoral contaxt of work 




Ufa skills orientation 




child oriantation 




1-3 Instructional LeveKs) of tha Materials 




Enqfish as a second tanquaqe (ESU 




basic literacy 




adult basic education (ABE) 




aeneral equivaJoncv diploma (GEO) 




ad rtneed skills 





Instructons for Types of CAI Packages 

Please check ONE box that best 
describes the purpose of the package or 
the subject component o^d multiple pur- 
pose package. 



Please check ONE box that best 
describes the context orientation of the 
package or the subject component of a 
multiple purpose package. 



Plea:*, check ALL relevant boxes that 
describe the instructional levels of the 
materials included in the package or the 
sublet component of a multiple purpose 
package. 



20 QUALITY MEASURES 



instructions for Quality Measures 

This evaluation form is intended to be used by trained eva uators to assess a CAI package used in the 
remediation of basic skills in JTPA programs. The CAI packages are evaluated in relation to the quality 
criteria presented in the following section. Each CAI package will be rated in terms of the extent to which 
each of the items spec Ted in this form are presented in the package being reviewed. For each item, the 
following scale is to be used: 

NA Not Applicable to this CAI package 
(A category to be used In rare Instances 
where an Item Is not an appropriate feature for the package.) 

Os Feature does not exist In the package 

1 * Feature exists but Is very poorly presented throughout 

2 a Feature Is presented but needs some Improvement 
3* Feature Is well presented 

4 a Feature Is presented exceptionally well throughout 

Please check only ONE box for each item 

Follow the instructions at the end of each section to calculate the Quality Rating 
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CAI Package Evaluation Form 



20 QUALITY MEASURES — Continued 



2.1 Curriculum Design Features 



Content 




u 


1 






f 


rvw/ttf vvU0 


lesson objectives are dearly defined 
















concepts are covered in depth adequate for the purpose of packaqe 
















content is consistent with the instructional levei 
















curriculum is a mastery based approach to fob related competencies 
















content includes testing to o arniert mastery of competencies 
















concepts are reviewed and reinforced 
















vocabulary, concepts, and examples are relevant to learner 
















lessons are manaqeabie In size 
















content b loqicai and wall organized 
















material reflects current knowiedae 
















content is grammatically error free 
















content is factually correct 
















to ml ai ts cnauenginq cm not irusiraung 
















content avoids ethnic, racial, sexual discrimination and stereotyping 
















Flexibility 


NA 


0 


1 


2 


3 


4 


Note Code 


difficulty level may be adjusted by learner and/or instructor 
















contains special items that can be modified by the instructor 
















Feedback 


MA 


n 
u 


i 






A 


hinta Coda 


immediate feedback to learner on performance 
















help routines are available for dues and examples 
















multilevel branchinq after wrono response to help routines and options 
















tone of address is appropriate to learners 
















feedback is useful and supportive of learninq 
















summary of learner performance is provided 


















NA 


o 


1 


2 


3 


4 


Note Code 


content coordinates and integrates oiner maienats in ine pacnage 
















content indudes visual rci/iforcement of learning 
















content indudes auditory reinforcement of learning 
















content indudes kinesthetic and tactile reinforcement of learning 
















Count Row (Count of checks for each column) 
















Ratlna Values 


NA 


XO 


X1 


x2 


x3 


X4 


Sum: 


Total Score Row ( Count x Ratlna Value) 


NA 


0 











Rating Calculation: 26 features— number NA = number Valid Features 
Sum of Ratings divided by Valid Features = Quality Rating for 2.1 
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CAI Package Evaluation Form 
20 Quality Measures— Continued 



2.2 Software Design Features 



Us§rlnt§rtac0 With Eaulpnwtt 


NA 


0 


7 


2 


3 


A 
H 


nolo UaJv 


colon selection and rang* of colors In the djaplav la effective 
















raaolutlon: images ara dear and detajjed _ 
















recognition: Imaqes ara reafotlo and easily racoqnlzad 
















complexity: a raasonabla numbar of Imaqaa ara disoiavad at ona tima 
















image size: physical siza of Images and characters la appropriate 
















animation: any movamant of Imaqaa supports learning 
















attraction: techniques to focus or sequence attention are effective 
















spaad: tima required to generate fanaoea ia raasonabla 
















sound Quality: sound la realistic 
















devices: use of headsets and speakers ara supported 
















adfustment learner can adjust tone and volume of sound 
















visual and audfrory material are consistent and complemetrtarv 
















keyboard: usa of keyboard is commensurate with level of Instruction 
















mouse: usa of a mouse ia supported for selection of menu options 
















liqht pen: use of a liqhtoan Is avaHable as an alternative to a mouse 
































khnuSvwtm*: 


NA 


0 


1 


2 


3 


4 


rVofe Com 


















dlalogua boxes ara used effectively 
















poo-up boxee are used effectively 
















menu displays can be toqqled on and off by leamer 
















movement among menus is simple and efficient 
















exit and escape routines ara available at all points 
















Deafen Component*: 


NA 


0 


1 


2 


3 


4 


rtota Coca 


«4Ifl/«fM«tlr> arv4 nJ*r*m*nt rnirtfnM are avaflahlA with oackaaa 
















unit planning assistance ia available based on diagnostic results 
















flexible lesson sequencing is supported 
















multilevel lesson sequences is supported 
















branchlnq to lessons at different level of difficulty is supported 
















items ara randomly selected for use in repeated lessons 
















Interaction With U**r: 


HA 


0 


1 


2 


3 


/* 


No f Coda 


a supportive and positive emotional climate is maintained 
















editing routines for leamer correction of errors are simple and effects 
















usa of help screens and explanations is appropriate and effective 
















usa of on-scraan directions is effective 




1 
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CAI Package Evaluation Form 



20 Quality Measures — Continued 



2,2 Software Ooslgn Features— Continued 

Functional Design Components— Continuad 



Infraction With Us*e— Continued: 



use of recorded voice instruction is affective 



routines are consistent and predictable 



NA 



Note Cad* 



responses to all user entries are immediate 



Meneaement Service* 



NA 



Not9 Cede 



lessons composing learning units are recorded and dlsplayed/printe i 



lessons assigned to individual learners are recorded/displayed/printed 



lessons completed by learners are recorded/dlsplaved/brinted 



learner performance on lessens Is racorded/dlsplayed/printed 



summary performance on learning units is recorded/dlsplayed/printed 



protest performance Is recorded/displayed/printed 



post-test performance is recorded/dlspiayed/brlnted 



time learners spend on lessons Is recorded/displayedfrrinted 



n umber of learner attempts per lesson Is recorded/displayed/printed 



Count Row (Count of checks for each column) 



Rating Values 



NA 



xO 



xl 



x2 



x3 



x4 



Sum: 



Total Score Row (Count x Rating Value) 



NA 



Rating Calculation: 43 features — number NA * number Valid Features 
Sum of Ratings divided by Valid Features = Quality Rating for 2.2 



ER?C 
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CAI Package Evaluation Form 



2.0 Quality Measures —Continued 



2.3 Implementation Features 



Installation 


NA 


0 


1 


2 


3 


4 


Note Code 


Installation ouide b comprehensive and dearly written with examples 
















installation programs allow options for storaqa locations 
















Installation procedures are simola and error free 
















minimum configuration requirements are dearly specified 
















peripheral hardware options are deanY specified 
















local area network options are available 
















Speed Of Operation 


MA 


n 

y 


l 






A 
*r 


riUW 


speed of opeartton is acceptable on Intended equipment 
















video display speed la reasonable on Intended equipment 
















data access spaed is reasonable on Intended equipment 
















any delay in operation b explained by a messaqe or Icon 
















Performance Options 


NA 


0 


- ~ 
1 


2 


3 


4 


Note Code 


ftftamtion b arrar frc* 
















iitjr ifinr ha nH linn anH raoowefV is *ff*criv* 
















nrintar nntfan* arm AVftllflhla and can ha chanced after installation 
















wwitww ftiAnitnF Afitlfina an <ti QpQftpH 
















common disk storaqa options are supported 
















common cursor control options are supported 
















Individualised student work disks ar a supported 
















multiuser options on local area network are supported 
















reasonable number of leamars are simultaneously suppported 
















reasonable number of learner records are maintained 
















Maintenance and Technical Support 


NA 


0 


1 


2 


3 


4 


Note Code 


documentation Is comprehensive and dearly written with examples 
















maintenance and update policy is reasonable 
















toll free telephone technical support is available 
















In-service traininq options are available 
















documentation Is comprehensive and dearly written with examples 
















Count Row (Count of checks for each column) 
















Ratino Values 


NA 


xO 




x2 


X3 


x4 


Sum: 


Total Score Row ( Count x Rating Value) 


NA 


0 








3 



Rating Calculation: 24 features — number NA a number Valid Feature* * 
Sum of Ratings divided by Valid Features = Quality Rating for 2.3 = 
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C AI Package Evaluation 

Form 



Description of the CAI Package 



Name: 



Contents of Package: 



Version: 



Publisher: 



CAI Package Code No: 
Subject Code: 
Hardware Requirements: 



Cost: 



Date of Review : 
Hardware Used for Review 



H dware Limitations: 



Reviewers: 

Quality Ratings: 

Curriculum Design Features: 
Software Design Features: _ 
Implementation Features: 



CAI Package Evaluation Form 



1.0 TYPES OF CAI PACKAGES 



1.1 Purpose of the Package 


driB and practice exercises 




supplemental instruction 




primary instruction 




simulation and application 




1.2 Context Orientation of the Package 




spactffc functional context of work 




aeneral context of work 




llfeskilta orientation 




chOd oriantation 




1.3 Instructional Level(s) of the Materials 




EnaHsh as a second lanouaae (ESL) 




basic litoracY 




adult basic education (ABE1 




Qeneral eouivaJency diploma (GED) 




advanced skills 


1 



Instructons for Types of CAI Packages 

Please check ONE box that best 
describes the purpose of the package or 
the subject component of a multiple pur* 
pose package. 



Please check ONE box that best 
describes the context orientation of the 
package or the subject component of a 
multiple purpose package. 



Please check ALL relevant boxes that 
describe the instructional levels of the 
materials included in the package or the 
subject component of a multiple purpose 
package. 



20 QUALITY MEASURES 



Instructions for Quality Measures 

This evaluation form is intended to be used by trained evakiators to assess a CAI package used in the 
remediation of basic skins in JTPA programs- The CAI packages are evaluated in relation to the quality 
criteria presented in the following section. Each CAI package wil! ha rated in terms of the extent to which 
each of the items specified in this form are presorted in the package being reviewed. For each item, the 
following scale is to be used: 

NA Not Applicable to this CAI package 
(A category to bo used In rare Instances 
where an Item is not an appropriate feature for the package.) 

Os Feature does not exist In the package 

1 m Feature exists but is very poorly presented throughout 

2 a Feature is presented but needs some improvement 
3« Feature Is well presented 

4 « Feature Is presented exceptionally well throughout 

Please check only ONE box for each item. 

Follow the instructions at the end of each section to calculate the Quality Rating 
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CAI Package Evaluation Form 



2.0 QUALITY MEASURES — Continued 



2.1 Curriculum Design Features 



uonjonj 




u 


1 




3 


A 


Note Cndn 


lesson objectives are clearly defined 
















concepts are covered in depth adequate for the nurpose of package 
















content is consistent with the instructional level 
















curriculum is a mastery based approach to job related competencies 
















content includes testing to document mastery of competencies 
















concepts are reviewed and reinforced 
















vocabulary, concepts, and examples are relevant to learner 
















lessons are manageable in size 
















content h logical and well organized 
















material reflects current knowledge 
















content is grammatically error free 
















content b factually correct 
















format is challenging but not frustrating 
















mrrtorrt ovmirt« athnir* rarifll 4AYUAI Hkt^rimtnaftnn and stereotvoinfl 
















Flexibility 


NA 


0 


1 


2 


3 


4 


Note Code 


difficulty level may be adjusted by learner and/or instructor 
















contains special herns that can be modified by the instructor 
















rvGGOaCK 


NA 


Q 


1 


2 


3 


4 


Note Code 


immediate feedback to learner on performance 
















help routines are available for dues and examples 
















multilevel branching after wrong response to help routines and options 
















tone of address is appropriate to learners 
















feedback is useful and supportive of learning 
















summary of learner performance is provided 
















t aarninn Stvte* 


NA 


o 


1 


2 


3 


4 


Note Code 


content coordinates and integrates other materials in the package 
















content memoes visual remiorcement or learning 
















content includes auditory reinforcement of learning 
















content includes kinesthetic and tactile reinforcement of learning 
















Count Row (Count of checks for each column) 
















Rating Values 


NA 


xO 


X1 


x2 


x3 


X4 


Sum: 


Total Score ROv ( Count x Ratine? Value) 


NA 


0 













Rating Calculation: 26 features— number NA = number Valid Features 
Sum of Ratings divided by Valid Features = Quality Rating for 2.1 
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CAI Package Evaluation Form 



2.0 Quality Measures— Continued 



2.2 Software Design Features 



color selection and range of cobra in the display Is effective 














1 


resolution: images am dear and detailed 
















recognition: imeges are realistic and easily recognized 
















complexity: a reasonable number of Images are displayed at one time 
















imaae size: ohvsical rize of imaqea and characters is appropriate 










— 






animation: any movement of imaaes supports learning 
















attraction: techniques to focus or sequence attention are effective 
















speed: time required to qenerate imaaes is reasonable 














~ — 


sound Quality: sound is realistic 
















devices: use of headsets and soeakers are supported 
















adtustment learner can adlust tor»~ and volume of sound 
















visual and auditory material are consistent and complementary 
















keyboard: use of keyboard is comme.aurate with level of Instruction 
















mouse: usn of a moua* is supported for selection of menu options 
















lioht pen: use of a liohtoen !* available as an alternative to a rr^use 
















runcuonai UQoign wwinp©* /ei i io 
















M*nu Systems.* 


NA 


0 


1 


2 


3 


4 


Not* Cod* 


full screen displays are used effectively 
















dialogue boxes are used effectively 
















pop-up boxes are used effectively 
















menu displays can be toqaleo on and off by learner 
















movement among menus is simple and efficient 
















exit and escape routines are available at all points 
















Denton Compcn — **• 


NA 


0 


1 


2 


3 


4 


Not* Cod* 


diagnostic rind placement routines are available with oackaae 
















unit planning assistance is available based on diagnostic results 
















flexible lesson sequencing is supported 
















multilevel lesson sequences is supported 
















branching to lessons at different level of difficulty is supported 
















items are randomly selected for use in repeated lessons 
















lnt*raction With (/sen 


NA 


0 


1 


*. 
s 


3 


4 


Not* Cod* 


a supportive and positive emotional climate is maintained 
















editing routines for learner correction of errors are slmplo and effects 
















use of help screens and explanations is appropriate and effective 
















use of on-screen directions is effective 
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CAI Package Evaluation Form 



2.0 Quality Measures— Continued 



□ 



2.2 Software Design Features — Continued 



Functional Design Comoouents — Continued 
Infraction With Usar— Continued: 



use of recorded vcice Instruction Is effectjye 



routines are consistent and predictable 
responses to all user enUes are Immediate 



Managamant Services: 

lessons composing teaming units are recorded and displayed/printed 
lessons assigned to individual learners are recorded/displayed/printed 



lessons completed by learners are recorded/displa red/printed 



learner performance on lessons is recorded/displayeoVbrinted 



summary performance on learning units is recorded/displayed/printed 



NA 



NA 



Not e Coda 



Note Coda 



protest performance is recorded/displayed/printod 



post-test performance * dcorded/displ^yed/printed 



time learners spend on lessons Is recorded/displayeoVprinted 



number of learner attempts per lesson is recorded/displayed/printed 



Count Row" (Cou^rt of checks for each column) 
Ratln Valuta 



NA 



xO 



Total Score Row ( Count x Rating ValueL 



NA 



x1 



x2 



x3 



x4 



Sum: 



Rating Calculation: 43 features — number NA = number Valid Features 
Sum of Ratings divided by Valid Features = Quality Rating for 2.2 
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CAI Package Evaluation Form 




Rating CaJcu'atlon: 24 features- number NA * number Valid Feature, 
Sum of Ratings divided by Valid Features * Quality Rating for 2.3 
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Chapter 3 

Evaluation Designs 



Two alternative evaluation designs are 
presented in this chapter. These evaluation 
designs are based on the information 
presented and discussed in the previous 
chapters. The designs were developed with 
careful consideration of current technical and 
research knowledge about the use of com- 
puter-assisted instruction (CAI) for the 
remediation of basic skills in JTPA 
programs. In the development of the designs, 
the Standards for Evaluations of Education- 
al Programs, Projects, and Materials (Joint 
Committee on Standards for Educational 
Evaluation, 1981) were carefully considered 
and applied to produce designs that meet the 
utility, feasibility, propriety, and accuracy 
standards established by the Joint Commit- 
tee. 

The National Commission foi Employ- 
ment Policy requested the developmer t of a 
large-scale, longitudinal design equal to ap- 
proximately six professional person years of 
effort and an alternative small-scale, survey 
type evaluation design. Both evaluation 



designs are structured to provide information 
about the effectiveness and efficiency of CAI 
packages used in the remediation of basic 
skills in JTPA piograms. The small-scale 
design retains the major sampling and 
analytic aspects of the large-scale design. 
However, for data collection Jhe small-scale 
design uses survey methods over a period of 
one year to obtain data while the large-scale 
design uses on-site data collection, observa- 
tions, and follow-up procedures over a two 
year time period. The large-scale design will 
be discussed in detail. The small-scale design 
will be described by indicating the specific 
design elements of the large-scale design that 
must be eliminated or al ered to reduce the 
scale of evaluation project 

For the purposes of the designs, the 
definition of CAI packages includes the 
software and all related instructional 
materials provided with the CAI package. It 
also includes the use of common materials 
and resources such as paper, pencils, cal- 
culators, dictionaries, maps, etc., incor- 
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porated by reference and instruction for their 
use in the CAI package. 

The term "traditional instruction" will be 
used in the designs to mean the various types 
of instructio. (such as lectures, films, 
workbooks, and projects) provided in JTPA 
programs that are not related to the use of a 
CAI package for the remediation of basic 
skills. This familiar term is preferred over the 
term "non-CAT because of syntax difficul- 
ties and awkwardness when the latter is used 



Traditional instruction does not mean to 
imply any negative values or lack of innova- 
tive effort associated with traditional 
methods Oi instruction. It is simply used as a 
term to differentiate other instruction from 
CAI. The limited research conducted by this 
project suggested that the evaluation design 
must anticipate that both CAI and traditional 
instruction will be used concurrently in most 
JTPA programs. 



Assumptions 



Many aspects of the operation of JTPA 
programs in terras of (a) recruitment and 
selection of participants; (b) methods of 
selection, purchase, and use of CAI pack- 
ages; (c) use of traditional instruction; (d) and 
hiring and training of instructional staff are 
beyond the control of the funding agencies. 
JTPA programs select and purchase CAI 
packages from a wide variety of vendors. 
Funding agencies do not control the selection 
and purchase process. Thus, the extant 
variety of CAI packages used in remediation 
of basic skills should be reflected in the 
evaluation. 

Many local factors affect the manner in 
which a given local program actually 
operates. Thus, evaluation designs should 
adapt to the natural setting as much as pos- 
sible and snould avoid an unrealistic clinical 
or laboratory model. 

The JTPA programs involved in the 
evaluation design will be actual on-going 
programs. The evaluation will interfere as 
little as possible with the normal operation of 
the programs. The evaluation should not 
cause substantial changes in the operation of 
existing programs. The manner in which ex- 



isting programs incorporate CAI packages 
and traditional instruction in the remediation 
of basic skills should be left intact 

The evaluation designs must follow 
sound research methodology to allow the 
results to be as generalizable as possible to 
the JTPA program population. The purpose 
is to obtain and evaluate information about 
the differential effects of CAI and traditional 
instructional modes and theirrespective cost- 
effectivenecs, rather than to evaluate the par- 
ticipating programs. Thus, it is important that 
the participating programs represent the 
population of programs as adequately as pos- 
sible. The participant must be representative 
of the participants found in programs 
throughout the population of JTPA 
programs. The manner in which CAI pack- 
ages are used in the design must be repre- 
sentative as possible of the actual and normal 
use of such packages in existing JTPA 
programs. 

The evaluation designs involve a variety 
of methods to assess the relative effective- 
ness and efficiency of CAI packages for the 
remediation of basic skills in job training 
programs. The designs evaluate and contrast 
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two instructional modes: computer-assisted 
instruction against traditional instruction, 
each in their natural program settings. The 
designs use existing program participants, 
instructors, teaching methods, curriculum 
materials, software, and equipment. The 
designs attempt to monitor and measure he 
differential effects of the instructional modes 
with as little inte mention and di ruption as 
possible. The program participants should 



not notice any difference in the services 
delivered during the evaluation from services 
delivered before and after the evaluation is 
completed. The program staff in participat- 
ing programs will be required to maintain 
and periodically report specific information 
on student performance and class participa- 
tion that may or may not be different from 
their normal record-keeping process. 



Large-Scale Design 



The primary question to be addressed by 
the evaluation designs is: 

How does the effectiveness and efficien- 
cy ofCAI packages compare with tradition- 
al instruction for the remediation of basic 
skills in JTPA programs? 

The effectiveness of the instruction 
modes will be determined by observing per- 
formance of JTPA participants on stand- 
ardized test of basic skills after competing a 
normal period of instruction in a JTPA pro- 
gram in one or both of the modes. The ef- 
ficiency of the instruction modes will be 
determined by contrasting the cost of obtain- 
ing a unit of increase in performance on a 
standardized measure of basic skills for CAI 
and traditional instructional modes. 

The evaluation will consider and address 
^veral specific issues reldted to the primary 
question. The issues relate to the extent to 
which various sets of variables contribute to 
explaining the variance in the performance 
outcomes in remediation of basic skills. The 
sets include variables relating to program 
type, participant chaiacteristics, instructor 
characteristics, time spent per instructional 



mode, and the CAI package characteristics. 
These variables exist in both traditional and 
CAI classes. Since JTPA programs are 
known to vary with respect to these areas, it 
is necessar* to include these variables and 
control tor their effects on performance out- 
comes in the assessment of the differential 
effects of CAI and traditional instruction. 

Otherrelated issues include a comparison 
of the effects of CAI and traditional instruc- 
tion on persistence or ietention rates of par- 
ticipants in JTPA programs. CAI may 
influence persistence more or tess than tradi- 
tional instruction. Student satisfaction wit 1 
instructional services may vary with CAI and 
traditional instruction. The opinions of the 
participants will be helpful in understanding 
how students relate to the two modes of 
instruction. Job placement rates for par- 
ticipants may be affected by the instructional 
mode. Experience with computers and 
software may be an advantage for JTPA par- 
ticipants seeking employment. The long- 
term effects of CAI need to be better 
understood. A follow-up job survey will 
determine the effects of the two instructional 
modes approximately one year after leaving 
JTPA training. The follow-up will attempt to 
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determine the extent to which the computer 
skills and software experience were relevant 
to the work place. 



Sample 



In the legislatively defined process fol- 
lowed in funding JTPA programs, funds flow 
from the federal government to the States and 
then to Service Delivery Areas (SDAs) 
which fund local JTPA programs. Because of 
the nature of the funding process, the most 
accurate information about current JTPA 
programs resides at the local level. Thus, 
considerable effort will be required in 
developing an accurate list of current JTPA 
programs and information about their use of 
CAI packages in the remediation of basic 
skills. Major CAJ package vendors are able 
to provide lists of JTPA programs that hold 
current licenses for use of their products. 
Some published lists of JTPA programs are 
available but are subject to becoming out of 
dat 5 rapidly and virtually useless. The appen- 
dix contains current JTPA programs located 
for purposes of this report 

Stratification 

Two primary strata must be adequately 
reflected in the sample: (1) programs which 
use CAI packages in remediation of basic 
skills and (2) programs which use only tradi- 
tional instructional methods. These strata are 
of primary interest. It can be expected that 
variances in variables of concern to the 
evaluation will differ between the strata. 

Other sampling concerns within each of 
these primary strata are representation for 
region of the U.S., typo of program., and type 



of CAI package used (drill, tutoring, primary 
instruction). 

CAI Stratum 

For the purpose of the evaluation designs, 
a JTPA program in the CAi stratum should 
have at least 6 months of experience with a 
CAI package to be considered eligible for 
participation as a site using CAI. This re- 
quirement will help to ensure that the local 
staff is beyond the initial training and im- 
plementation phase. It is expected that these 
programs will also use varying degrees of 
traditional instruction. In the CAI stratum, 
the CAI packages used in the remediation of 
basic skills will be the software selected, 
purchased, ana :n use by the local programs 
prior to the evaluation. A viriety of CAI 
packages should be. expected to be in use. 
More than one CAI package may be used \i 
a single program with a single participant 
The evaluators will use the software evalua- 
tion form developed in this project and dis- 
cussed in Chapter 2 to score all of the CAI 
packages being used in the CAI group on 
both the curriculum design factors and 
software design factors. These scores will be 
used in the analysis as variables in explaining 
the effectiveness of the CAI packages. A 
procedure is provided to account for use of 
multiple packages with a single participant 

Traditional Instruction Stratum 

A JTPA program in the traditional in- 
struction group should not be using CAI at 
all. These programs should represent the 
traditional instructional methods as clearly as 
possible. 
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RdfidOtn Sampling Of 

JTPA Programs 

A list of all known JTPA programs in eac h 
stratum should be developed A random sam- 
pling procedure should be conducted within 
each strata to select JTPA programs for par- 
ticipation in the evaluation. The randomly 
selected JTPA programs must be contacted 
and considerable effort expended to secure 
their commitment to participate in the 
evaluation. The need to continue the random 
sampling procedure to replace programs 
which elect not to participate should be an- 
ticipated 

The A . ^grams selected in the CAI group 
should be interviewed for potential problems 
with their use of tiie CAI package that would 
jeopardize their full participation in the 
evaluation. These programs should be using 
CAI in their regular and normal operation 
and should be beyond the training, introduc- 
tory, or transitional use of CAI. Programs 
with potential problems that would limit full 
participation or with abnormal CAI opera- 
tional problems should be replaced. A record 
of the reasons any program was rejected from 
the sample should be kept and reported in the 
final report 

JTPA Program Sample Size 

Both strata must be well represented in the 
sample. Because the focus is on comparisons 
of the strata h terms of performance in 
remediation of basic skills, near equal num- 
bers of programs for each strata should be 
obtained. It is recommended that 100 
programs be included in the total sample, 
with approximately 50 JT?A orograms in 
each of thu primary stratum. 



Number of Participants 
in the Sample 

A random sample of participants or stu- 
dents in the programs selected and agreeing 
to participate will determine which students 
participate in the evaluation. The eligible 
participants would be only those students 
^ceiving instruction by the program for the 
leraediation of basic skills and whose perfor- 
mance or record can be tracked for at least 
one year. Students not receiving remediation 
services but involved in other aspects of the 
JTPA program would not be eligible par- 
ticipants. 

A random sample of 20 participants from 
each program will provide approximately 
1,000 cases for each stratum and a total of 
2,000 cases for the evaluation. The 20 cases 
will not create an unreasonable record-keep- 
ing burden on the individual projects. Of 
course, the proportion of participants in- 
volved (and the probability of being random- 
ly selected) will vary somewhat depending 
on the number of students in remediation of 
basic skills in a project. The potential in- 
dividual selection bias involved due to dif- 
ferences in number of eligible students is 
outweighed by the concern for maintaining a 
reasonable record-keeping burden on the 
local programs. If more than 20 cases per site 
are requested, the additional record-keeping 
burden could result in a potential for inac- 
curate data. 

In all programs, the selection of the 20 
participants per program must be random 
except in small programs where all par- 
ticipants in the program must be included in 
order to met the quota. Particular participants 
must never be purposefully selected to par- 
ticipate or excluded from participation. 
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Method 



The method planned for the evaluation is 
outlined below. There are five outcome 
measures which are analyzed in relation to 
the two instructional modes, CAI and tradi- 
tional instruction. Control variables are used 
to neutralize the effect of various program, 
participant, and instructor variables known 
or expected to have significant effect on the 
outcome measures. In the following sections, 
the criterion and predictor variables will be 
presented and discussed The data collection 
processes will be described The use of data 
colfr :tion forms and survey instruments to 
collect information on the variables will be 
discussed 

The primary analytic framework will be a 
pre-test, treatment, post-test design. The data 
will be analyzed by hierarchical model mul- 
tiple correlation regression analyses to con- 
trol for multicollinearity among the 
variables. The techniques required to per- 
form the analyses will be c -cussed in detail. 
The format of suggested tables and graphs 
will be presentee 1 o aid in the interpretation 
of the findings. The analysis plans for each 
criterion variable will be described. 

Criterion Variables 

A series of outcome measures will be 
used in the evaluation to measure perfor- 
mance outcomes that may be influenced by 
the type of instruction provided. These out- 
come measures are direcdy mlated to the 
objectives of the JTPA programs. The out- 
come measures represent the criterion vari- 
ables that will be considered in the 
evaluation. Five criterion variables will be 
considered: 



• Post-Test Score on the Test of Adult 
Basic Education (TABE) 

• Persistence (Retention) Rate of Par- 
ticipants 

• Student Satisfaction Rating of Instruc- 
tional Services 

• Job Placement Rate for Participants 

• Follow-up Job Survey Results 

Test of Adult Basic Education (TABE) 

The Test of Adult Basic Education 
(TABE) is a standardized test used to 
measure participant performance in the basic 
skills. The test provides scores for reading, 
mathematics, and a total score. The test is 
used widely in JTPA training programs and 
adult education programs to assess perfor- 
mance in basic skills. The validity of the 
TABE has been well documented, and the 
results of die TABE correlate well with other 
measures of performance in basic skills. It 
has been used extensively in programs with 
relatively brief periods of instruction, a com- 
mon practice in JTPA programs. A set of 
pre-test scores in reading, mathematics, and 
for the total test using the TABE will be 
required for all participants and used as con- 
trol variables in the analyses. Post-test scores 
will be used as criterion variables. 

Persistence Rate 

The Persistence (or retention rate) of Par- 
ticipants is an indicator of the value that the 
students place on participating in the pro- 
gram relative to other alternatives and uses 
of their time and effort. Experience with 
computers and CAI software may be per- 
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ceived to be more valuable to some students 
and result in higher persistence rates than 
traditional instruction. Students may (or may 
not) find the CAI more interesting and 
motivating than traditional instruction. 

Student Satisfaction Rating 

The Student Satisfaction Rating is an 
opinion expressed by the students regarding 
the instruction provided by the program. Stu- 
dents may have different levels of satisfac- 
tion with the CAI and traditional methods of 
instruction. The measure of satisfaction will 
be determined by a Questionnaire or inter- 
view schedule. The primary issues addressed 
will be participant opinion regarding issues 
such as quality, pace, motivation, value, ap- 
propriateness, and relevance of the instruc- 
tion provided through CAI and/or traditional 
instruction. 

Job Placement Rate 

Differences in job placement rate for par- 
ticipants may be related to the participants' 
experience using computers and CAI pack- 
ages. Job placement rate may be related to 
the level of performance attained by tradi- 
tional or CAI instruction. 

Follow-Up Job Survey 

Students in both instructional groups will 
be tracked for at least one year. A Follow-up 
Job Survey will be used to determine the 
influence of the instructional method on get- 
ting the job, keeping the job, performance on 
the job, and perceptions of value of instruc- 
tional methods in helping them succeed after 
leaving the program. 



Predictor Variables 

The predictor variables will include sets 
of variables expected to be related to dif- 
ferences in the outcome measures: the char- 
acteristics of the program, participants, 
instructors, time spent in instruction, and 
CAI p. kages. 

Program Variables 

The program variable set controls for the 
effects of type of program (Block Grant, 
Summer Youth, etc.) and class size. Some of 
the variance in the outcome measures is ex- 
pected to be explained by or associated with 
these variables. 

Program Type Set (XI) 

Program type set of variables refers to the 
various types of JTPA programs. These are 
categorical variables and are operationalized 
by using effects coding (Cohen & Cohen, 
1975, p. 188) with the Title IV Federal Pro- 
gram used as the base from which to com- 
pare. The program types included in the 
analysis are given below with the variable 
name used in the equations: 



Piofram TvDe 


Xla 


Xlb 


Xlc 


Title H-A 


1 


0 


0 


Title H-B 


0 


1 


0 


Title HI 


0 


0 


1 


Title IV 


-1 


-1 


-1 



Class Size (X2) 

The class size is the average number of 
students in JTPA basic skills classes in- 
volved in the evaluation. This variable helps 
to characterize the structure of the JTPA class 



ERIC 



77 



64 



in which the instruction is delivered. It is a 
numeric variable. 

Participant Variables 

The participant variable set allows the 
analysis to determine the effect of the char- 
acteristics of the students involved in the 
evaluation. Some of the variance in the out- 
come measures is expected to be explained 
by or associated with these variables. 

Language Group (X3) 

The language groups are English as a 
second language (ESL) and native English 
speaking students. This variable is a categori- 
cal variable that will allow the analysis to 
determine the effect of language groups on 
performance in basic skills using the two 
instructional methods. The variable is coded 
as follows using dummy coding: 

Language Group X3 
ESL 1 
Native English 0 

Age (X4) 

Age of the participant will be entered as a 
characteristic of the participant to determine 
the effect of age on performance in basic 
skills for each instructional mode. Age is a 
numeric variable. 

Schooling (X5) 

Schooling of the participant will be 
entered as a characteristic of the participant 
to determine the effect of number of years of 
formal education on performance in basic 
skills for each instructional mode. Schooling 



is a numeric variable representing the num- 
ber of years of formal education. For ex- 
ample, a participant who completed the sixth 
grade and dropped out of school in the 
seventh grade would h*ve six years of educa- 
tion. This /ariable does not attempt to in- 
clude any measure of quality or effectiveness 
of the number of years of schooling. The 
pre-test TABE scores discussed below are 
used to measure and control for current 
academic ability. 

Work Experience (X6) 

Work experience of the participant will be 
entered as a characteristic of the participant 
to determine the effect -f number of years 
work experience on performance in basic 
skills for each instructional mode. Work ex- 
perience is a numeric variable representing 
the number of years of work experience. 

TABE Pre-Test Scores (X7a and X7b) 

The pre-test scores for reading (X7a) and 
mathe matics (X7b) will be used to control for 
the effect of the participant's academic 
ability in basic skills at the beginning of the 
period of instruction in either modes. Using 
the pre-test scores as control variables avoids 
the serious problems associated with the use 
of gain scores (Cohen & Cohen, 1975, pp. 64 
and 67-63). The scores are numeric values 
that can be considered analytically as a set or 
individually. 

Instructor Variable Set 

The instructor variable set controls for the 
effects of the skills, characteristics, and 
qualities of the instructor on the participants 
performance in basic skills. The research 
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conducted in the design project found that the 
expert panel members and the service 
providers tend to perceive the instructor to bt 
very important to the success of CAL Some 
of the variance in the outcome measures is 
expected to be explained by or associated 
with the instructor variables. 

TESA Score (X8) 

Phi Delta Kappa, a professional education 
association, publishes, an instrument and 
training program, Teacher Expectations and 
Student Achievement (TESA). The TESA 
score focuses on teacher behaviors and class- 
room skills that encourage effective teaching 
practices with students who are perceived to 
be low achievers. The TESA program has 
been widely used and nationally validated. 
The TESA interaction model has been shown 
to be particularly effective in improving the 
quality of teaching of low achieving students. 

The complete TESA scoring system 
focuses on high and low achieving students 
and the differences in teacher behavior be- 
tween these groups. It is too extensive for the 
needs of the current evaluation, although it 
would likely be of benefit to some instructors 
in JTFA programs. For the purposes of the 
evaluation, the TESA interaction model 
presented below is of particular importance 
because it establishes a set of instructor be- 
haviors that can be observed and rated as 
detailed in the TESA instructions and 
manuals. 

STRAND A: Response Opportunities 

• Equitable Distribution: Distributes 
response opportunities equitably versus 
unreasonably prohibits student from 
responding 



• Individual Help: Gives student in- 
dividual assistance versus ignores a re- 
quest or signal for individual heJp 

• Latency: Allows time for responding 
versus terminates response opportunity if 
response is net immediate 

• Delving: Delves, rephrases, gives ciues 
versus does not delve afterunsatisfactory 
response 

• High Level Questions: Uses higher 
le\ Questioning; calls for student 
opiiuwti, explanation, evaluation versus 
teacher suggests degree of difficulty as- 
sociated with question 

STRAND B: Feedback 

• Affirm/Correct: Affirms or corrects 
student's performance versus does not 
react or comment after student's 
response 

• Praise: Praises student' s learning perfor- 
mance versus criticizes; goes beyond 
simply correcting or negating 

• Reasons for Praise: Gives reason for 
praising student's learning performance 
versus is sarcastic or gives insincere 
praise 

• Listening: Listens attentatively to stu- 
dent versus is inattentive 

• Accepts Feelings: Accepts and reflects 
student's feelings in non-evaluative 
manner versus discourages or disparages 
student's feelings 

STRAND C: Personal Regard 
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• Proximity: Moves within arm's reach of 
student versus isolates student 

• Courtesy: Expresses courtesy versus 
makes rude or insulting remarks to stu- 
dent 

• Personal Interest and Compliments: 

Takes personal interest in student or 
gives compliment versus curtails or belit- 
tles student's personal statement 

• Touching: Touches student in a friendly 
manner versus rejects student's attempt 
to touch; slaps, grabs, touches angrily 

• Desiat: Corrects student behavior in a 
calm, courteous manner versus teacher 
displays hostility, unreasonable anger 

The 15 elements of the- interaction model 
establish a set of teacher behaviors and skills 
that are considered important for quality in- 
struction and a set of negative or opposite 
behaviors that should be avoided Through a 
series of five 30 minute observations, the 
instructor can be scored with simple counts 
on all fifteen items in the three stands 
described above. The score measures the ex- 
tent to which the teacher exhibits the positive 
and negative behaviors during instruction. 
The local numeric scores can be compared 
with regional and nadonal level data main- 
tained by Phi Delta Kappa. For the evalua- 
tion, the ^core would be used as a numeric 
measure of instructor quality for both CAI 
and traditional instruction. 

Teaching Experience (X9) 

Years of teaching experience will be 
entered as a characteristic of the instructor in 
order to determine the effect of number of 



years teaching experience on performance in 
basic skills for each instructional mode. 
Teaching experience is a numeric variable. 
Teaching experience was selected as a more 
relevant variable for this evaluation than for- 
mal education because of the differences in 
the instructional methods. CAI is a relatively 
new approach with very limited opportunity 
for relevant formal training in direct contrast 
to extensive opportunities for formal educa- 
tion in traditional instructional methods. 
Since most teachers are required to be cer- 
tified » nd have completed very similar for- 
mal -educations, regardless of the 
instructional mode, the formal education was 
considered of little importance in the present 
evaluation. However, teaching experience 
would be helpful to instructors using either 
mode of instruction. The amount of teaching 
expeiience instructors have can vary widely. 

Time Variables 

A critical set of variables concerns the 
amount of time tl?at the participants spend in 
each instructional mode. Time is anticipated 
to have an important effect on performance 
for both CAI and traditional instruction. 
Time is an important element in determining 
the relative effectiveness of the two instruc- 
tional modes. 

Total Ti^ein CAI (X10) 

Total time in CAI represents the total time 
measured in hours that a participant is in- 
volved in receiving instruction, using com- 
puters, or using materials associated with a 
particular CAI package during the student's 
participation in the JTPA program. Separate 
time records will be kept for each CAI pack- 
age the participant uses in the manner 
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prescribed on the data collection form for 
calculation of quality interactions used in the 
regression analyses. Total time in CAI is a 
numeric measure that will be used in con- 
junction with CAI quality measures in deter- 
mining the effect of CAI instruction on 
performance in basic skills. 

Total Time in Traditional (Xll) 

Time in traditional instruction for par- 
ticipants in either the CAI stratum or the 
traditional stratum will be measured as hours 
of lecture and of individual, group, or class 
discussion, desk work, or activities iiat have 
no association with CAI. 

Total Time 

TotaJ time is the computed sum of time in 
CAI and lime in traditional. This variable can 
be computed for supplementary analysis of 
the overall impact of instructional time on 
performance in basic skills. Since it is a direct 
function of the two variables, it will not be 
used in the regression analyses. 

CAI Package Variables 

The CAI package variables will be ob- 
tained from the evaluators use of the CAI 
Package Evaluation Form presented in Chap- 
ter 2. Each CAI package used in the evalua- 
tion must be rated using the form by trained 
evaluators. The scores determined for a par- 
ticular package will be used consistently 
throughout the evaluation and applied to 
each case in which a student uses that pack- 
age in any JTPA program. The results of the 
assessments of the CAI packages will not be 
made known to the local programs until the 
completion of the evaluation. At present it is 



not known what relationship the rating will 
have v/ith performance in basic skills. Im- 
plications that a particular CAI package is 
"high quality" and another is not could bias 
the evaluation anc limit the number of CAI 
packages used in the evaluation. 

CAI Package Codes 

Each CAI package rated with the evalua- 
tion form will be given a code number by the 
evaluators for use on the data collection 
forms. The results of the quality assessment 
of the CAI packages will be kept confidential 
until after the evaluation is completed. The 
codes will be used by the evaluators to iden- 
tify the packages with the rating scores at the 
time of analysis. The CAI package code is 
not an analytic variable but simply a means 
of identifying the packages. It will be used to 
identify specific packages in the descriptive 
analyses which sort on the most effective and 
efficient CAI packages for various groups of 
participants. 

Subject Code 

Each CAI package will be given a subject 
code that indicates the primary basic skills 
focus of the package in terms of reading and 
mathematics. This variable will not be an 
analytic variable but another means of class- 
ifying the packages. 

CAI Type Variables 
(X12a X12b X12c X13a X13b X13c) 

The CAI type variables will be entered in 
the analyses using three set of analytic vari- 
ables used to classify packages by their in- 
structional purpose as described in detail in 
Chapter 2. The classification will be made by 
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the trained evaluator and recorded on the CAI 
package evaluation form. An effects coding 
technique will be used to code the variables 
for the regression equations. The coding for 
the purpose of the package is presented 
below. The effects coding scheme contrast 
each group with the entire set of groups. 

Purpose X12a X12b X12c 

Drill and Practice 1 0 0 

Supplemental 0 1 0 

Simulation 0 0 1 

Primary Instruction -1 -1 -1 

The coding for the context orientation of 
the package is presented below. 

Context X13a X13b X13c 

General Work 1 0 0 
Life Skills 0 1 0 

Child Orientation 0 0 1 
Specific Work -1 . -1 -1 

There may be more than a single instruc- 
tional level in a package. The data will be 
included in the data collection process using 
the CAI package evaluation forms but will 
not be included in the regression analysis. 
These data can be used in descriptive 
analyses of the packages. 

CAI Quality Variables (Xi4 X1S X16) 

Three quality rating variables will be 
produced by assessing the CAI package 
using the CAI package evaluation form. The 
variables are a numeric measure of three 
aspects of quality of CAIpackages. The three 
quality variables will be mean quality ratings 
for all CAI packages used by an individual 
participant. The time spent per CAI package 
will be multiplied by the quality rating to 



obtain quality points. Total quality points 
divided by total time in CAI yields a mean 
quality rating for the participant who uses 
more than one CAI package during the 
evaluation. 

For each CAI package the following three 
sets of calculations are made: 

1. Time in CAI package x 
Curriculum Design Rating = 
Curriculum Quality Points 

Total Curriculum Quality Points / 
Total Tims in all CAI packages = 
Mean Rating for Curriculum 
Design Features 

2. Time in CAI package x Software 
Design Rating = Software Quality 
Points 

Total Software Quality Point: / 
Total Time in all CAI packages « 
Mean Rating for Software Design 
Features 

3. Time in CAI package % 
Implementation Design Rating = 
Implementation Quality Points 

Total Implementation Quality 
Points / Total Time in all CAI 
packages = Mean Rating for Im- 
plementation Design Features 



The mean quality ratings will provide 

X14 Mean Rating for Curriculum 
Design Features 
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X15 Mean Rating for Software Design 
Features 

X16 Mean Rating for ^mplementaton 
Features 

T.«* CAI quality variables are of major 
importance in the evaluation and will be in- 
cluded in the equations as main ef^cts and 
in interactions with time in CAI- These inter- 
actions represent the total iarty points 
function and permit the analyses to consider 
the effect of a CAI package of a certain 
quality on performance at varying amounts 
of time. 



Data Collection Methods 



The evaluators will provide detailed in- 
structions and forms to the participating 
projects to report data for each of the 20 
randomly selected students participating in 
the evaluation. The data will be aggregated 
by the project and mailed to the evaluators. 

The evaluators or trained special person- 
nel will make site visits to program sites for 
monitoring the da'a collection process. They 
will spot check iht record-keeping process 
and verify accuracy of the data. The 
p jgrams will administer the required pre- 
test and post-test using the TABE. The 
programs will administer the satisfaction sur- 
vey instruments to their respective par- 
ticipants. Information on persistence and job 
placement will be maintained and provided 
by the programs on the cvaluator forms. The 
programs will provide information about 
employment to the evaluators who will make 
the one year follow-up contacts and ad- 
minister the follow-up survey. 



During the site visits, the evaluators or the 
trained special personnel will gather data for 
each of the predictor variables not included 
on the fc : and additional qualitative infor- 
mation about the use of CAI packages xor 
remediation of basic skills . They will observe 
the use of the CAI packages and interview 
students and staff involved in the use of CAI 
packages. They will observe the instructors 
in both CAI and traditional instruction modes 
and use the TESA instrument to obtain the 
instructor ratings. The evaluators will gather 
the necessary data for the cost variables for a 
cost-effectiveness analysis detailed below. 

The trained evaluators will us, CAI 
evaluation form on site or in the e\ Uuators 
office to assess *e CAI packages used iii the 
JTPA programs and obtain the CAI type 
informal' un and the quality ratings. A given 
CAI package need only be assessed once. 
The ratings for a CAI package will be used 
consistently throughout the evaluation. The 
evaluators wiT . ermine a method of using 
consensus of a panel or a series of evaluators 
to assess the packages to obtain valid and 
reliable ratings for the packages. 



Data Analyses 



The statistical procedures planned to 
analyze the data are outlined below. The 
procedures will require the use of a 
mainframe computer and statistical packages 
such as the Statistical Package for Social 
Sciences (SPSS-X) or the Statistical 
Analysis System (SAS). 
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Multiple Regression 
Correlation Analyses 

Hierarchical m'xiel correlation regression 
analyses will be used to analyze the reading 
and mathematics criterion variables in rela- 
tion to the predictor variables described 
above. Four regression analyses are required. 
For the data from programs which use both 
CAI and tradition instruction, one regression 
analyses will aralyze the data in relation to 
the reading criterion varuuie and another 
will analyze the data in relation to the math- 
ematics criterion variable. 

Another pair of rr session analyses will 
be required for those programs having only 
traditional instruction. Of course, in these 
analyses, all variables relating to CAI will be 
deleted. Otherwise, the analyses will proceed 
in the same manner as described below, 

A comparison of the results between the 
two pairs of analyses will provide more in- 
sight into the effects of CAI. The traditional 
only instruction group will provide a base 
line with which the CAI instruction can be 
compared 

There will be approximately 1,000 cases 
used in each of the two pairs of analyses. 
Therefore, the statistical power will be ade- 
quate to test the variety of predictor variables 
and interactions included in the equation, 

Multicollinearity 

The predictor variables are expected to be 
correlated to some degree due to the existing 
nature of the JTPA programs. This multicol- 
linearity requires that a model be established 
that defines the order in which the variables 
are entered into the equations. The use of 
hierarchical model correlation regression 



analyses will permit each variable or set of 
variables to be tested for statistical sig- 
nificance in explaining incremental variance 
in the criterion variable as the set is entered 
into the equation (See Cohjn & Cohen, 1975, 
for a thorough discussion o c the rationale of 
this procedure). 

Procedures 

The data must be entered into data bases 
such that all variables can be accessed using 
the SAS ur 3PSS-X programs on a 
mainframe computer. Tne data should be 
carefully checked and verified for accuracy 
before the analysis is conducted. Missing 
data elements will mean that the case is ex- 
cluded from the analyses. Efforts should be 
made to obtain complete data for all par- 
ticipants in the evaluation. Initial descriptive 
statistics should be obtained for each variable 
and carefully studied to detect oudiers and 
possible data entry errors. Obvious data 
problems in cases should b; reconciled and 
corrected or the case should oe eliminated so 
that it & c* not distort the analyses. 

Regression Equations 

The « uiables used in the regression equa- 
tions have been defined and discusse x above. 
The following equations will be used in two 
regression analyses, one for reading and one 
for mathematics, using the same predictor 
variables. 
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Post-Test 
TABF Score 
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Constant 




1 . Program Set 


+ Xid + Xlb 




+ Xlc 


Program Type Set 


+ X2 
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+ X3 
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+ X7a + X7b 
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Total Time in CAI 


+ X11 


Total Time in Traditional 
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+ X12c 
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Context Orientation 
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Curriculum Design Rating 
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Implementation Rating 




Traditional I r fractions 


+X8*X11 


Time in Traditional 




* ILoA ocore 




CAIOualitv Interactions 
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Time in CAI 




* Curriculum Rating 


+X10*X15 


Time in CAI 




* Software Rating 


+X10*X16 


Time in CAI 




^Implementation Rating 



+X10 * Xl2a 
+X10 * X12b 
+X10 * X12c 

+X10 * X13a 
+X10 * X13b 
+X10*X13c 



+X8 * X14 
+X8 * X15 
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CAfType Interactions 
Time in CAI 
* Purpose Set 



Time in CAI 

* Context Orientation 



CAI Quality andTESA 

Interactions 

TESA Score 

* Curriculum Rating 
TESA Score 

* Software Rating 
TESA Score 

* Implementation 

Rating 



Testing the Main Effects 

The main effects of the predictor variables 
will be tested by first running a protected 
F-test to determine if the overall equation 
explains a statistically significant amount of 
the variance in the criterion variables. The 
.01 level of statistical significance will be 
used The test of the main effects will be 
performed by entering the predictor variables 
in sequence indicated by th^ equation vari- 
able number (X3 then X4). Variable sets such 
as Xla, Xlb, and Xlc will be entered at one 
step. At each stsp the added increment of 
variance in the criterion variable explained 
by or associated with the added predictor 
variable w: U be tested for significance using 
an appropriate F-test (Cohen & Cohen, 1975, 
p. 136) 

The results w:ll indicate the extent to 
which each variable is able to explain dif- 
ferences in basic skills performance beyond 



erjc 



85 



72 



the differences accounted for by the variables 
already in the equation. This process controls 
for the effects of previously entered variables 
and removes the problem of multicollinearity 
from the interpretation of the results. 

The interpretation of the categorical vari- 
ables will be a contrast with the entire set of 
groups (Cohen & Cohen, 1975, p. 191). The 
raw regression coefficients indicate the ef- 
fects of the groups. 

Testing the Interactions 

A series of interactions are tested next. An 
interaction of time in traditional instruction 
andTESA score is tested If it is significant, 
the interaction should remain in the equation. 
It will indicate whether or not the effect of 
time in traditional instruction on basic skills 
performance is dependent upon various 
levels of the TESA measure cf teacher 
quality. 

The CAI quality interactions are tested 
next There are three separate interactions to 
be tested in the sequence given. These inter- 
actions will determine if the effect of time in 
CAI on basic skills performance varies with 
various levels of CAI package quality in each 
of the three areas assessed. It is suggested that 
the measures be entered in the order ^iven. 
An interaction is determined to be significant 
when the incremental variance is determined 
to be statistically significant If the interac- 
tion is not significant, it may be dropped from 
the equation prior to testing the next interac- 
tion. All main effects involved in any inter- 
action deluded in the equation must be 
retaineu and entered prior to the interaction. 

Next, the interactions of time in CAI and 
purpose of the CAI package and the interac- 
tion of time in CAI and the context orienta- 
tion of the CAI package are tested. These 



interactions will determine whether or not 
time and type oi package interact in the effect 
on performance. 

The last interaction test is the TESA Score 
ana u*c CAI quality measures. These interac- 
tions determine if the teacher quality inter- 
acts with the CAI package quality in relation 
to the performance measures. 

Interpretation of Results 

The results of the regression analyses 
should be interpreted with care. The results 
of the regression analyses should be dis- 
played in a table showing the variables 
entered at each step and the increm' in 
R-squared, the squared multiple collation 
coefficient, which indicates the added per- 
centage of variance in performance ex- 
plained by a variable or sit of variables added 
at that step. For each increment, the degrees 
of freedom, F-test, and probability should be 
given. 

The interactions must be tested prior to 
interpreting the main effects. A significant 
interaction means that the main effect in- 
volved in the interaction cannot be inter- 
preted correctly in isolation from one 
ancher. Thus, the main effects are meaning- 
less and the interpretation of the variables 
must be made through the interaction. 
Graphs should be used to illustrate the results 
of all interactions found to be significant. The 
production of the graphs requires that the 
regression equation be entered into a 
micxcccinputer spreadsheet program (such 
is Lotus 1-2-3). The equation is solved using 
means for all control variables and selected 
valuer for the main effects being studied The 
process ailows various situations to be 
entered and liustrated, given the effects of 
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each varable as determined by the regression 
analysis. 

Cost-Effectiveness Analysis 

The cost variables collected using the CAI 
package evaluation form and on-site data 
collection activities will provide data that 
may be compared with the outcome perfor- 
mance measures. The cost elements impor- 
tant to the analysis are: 

• CAI Package Costs: The cost of each 
CAI package used in the JTPA prognm. 
These are fixed direct costs. 

• Hardware Costs: The cost of all com- 
puter hardware used in CAI in the JTPA 
program. These arc fixed direct costs. 
There is little if any salvage value or 
trade-in value for used computer equip- 
ment because it becomes obsolete rapidly 
and the cost of comparable new equip- 
ment tends to ./e less than what was paid 
for the old equipment 

• Maintenance Costs: The cost of main- 
taining the hardware and the CAI pack- 
age used in the JTPA program. These are 
variable direct costs that may increase as 
a function ""hours of use and the age of 
the hardware. 

• Training Costs: The cost of training in- 
structors to use the CAI package are vari- 
able direct costs depending upon the 
turnover of personnel. 

• Instructor Costs: Tht personnel cost for 
instructor(s) are variable costs that are a 
function of hours of use. These costs 
should include salary or wages and as- 
sociated fringe benefits. 



The fixed costs can be spread over the 
total hours of use during the expected life of 
the CAI package and hardware for an es- 
timated cost per hour of using the CAI pack- 
age. The more the package an/ computer 
system are used, the more benefit is received 
from the investment. However, the recurring 
costs increase with use as a function of hours 
of use of the CAI package and hardware* A 
comparison of the cost per unit of output in 
terms of basic skills performance for CAI 
vers: ^aditional instruction will provide an 
analysis of the cost-effectiveness of the CAI 
and traditional instruction modes. 

Cost-efficiency can also be considered. 
The cost per hour of operation may be calcu- 
lated for the CAI package using thest cost 
elements. The productivity of the CAI pack- 
age per hour will be determined by the 
regression analyses, controlling for various 
conditions and variables in the JTPA pro- 
gram. The productivity is measured in in- 
creases in performance in standardized 
measures of basic skills using the TABH. 
Thus, the cost-efficiency of the CAI package 
can be determined per productivity unit per 
hour. 



CAI Package Cost 
h Hardware Cost 
— Salvage Value 

= One-Time Cost per 

Life-Time Hours Hour of Use 



■ Estimated Hourly Training Cost 
+ Estimated Hourly Maintenance Costs 
+ Hourly Instructor Cost 



Total Hourly Cost of CAI 
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Cost-efficiency = 



Productivity per Hour 
Total Hourly Cost of CAI 



A similar analysis of the cost of traditional 
instruction can be compared to the cost-ef- 
ficiency of the CM package. In the tradition- 
al instruction mode, the computer related 
costs will not be considered, of course, but 
the costs of other instructicral materials such 
as books and supplies must replace them. 
Equipment used in the traditional instruction 
should also be considered in the same manner 
as the computer equipment has been con- 
sidered in the above structure. The regression 
equation using the traditional only instruc- 
tion group will provide the productivity 
measures for use in this cost-efficiency 
analysis. 

Survey Analysis 

The results of the surveys should be used 
to make comparison of overall instructional 
group effects on persistence and placement 
rates. A t-test should be conducted to deter- 
mine the statistical significance of the dif- 
ference be; ween mean persistence rate and 
mean placement rate for Traditional Instruc- 
tional Group participants and CAI Group 
participants. 

A comparison should be made of the 
results on items on the Satisfaction Survey 
and the Follow-up Job Survey Results for 
two instructional groups. Where applicable, 



the results need to be presented in tables and 
graphs. Numeric data should be tested for 
statistical significance with t-test or analysis 
of variance. 

Descriptive Statistics 

Descriptive statistics on the variables col- 
lected throughout the evaluation should be 
performed and reported in tabular form. 
These data will be important in describing the 
sample and the nature of CAI and traditional 
instruction in JTPA programs. 

Qualitative Analysis 

The evaluation should include a qualita- 
tive or case study approach that collects ade- 
quate descriptive data to permit a valid 
narrative portrayal of the use of CAI and 
traditional instruction in JTPA programs. 
This analysis should include a careful 
description of the way in which CAI is im- 
plemented and used in JTPA programs. 1c 
should address the problems and success that 
various programs have experienced in using 
CAI instruction. The qualitative study should 
consider how the use of CAI and traditional 
instruction are combined in those programs 
that use both men* %# Consideration should be 
given to determining why some programs 
use CAI and why others do not The level of 
training and experience of the local staff and 
their attitudes toward CAI are important is- 
sues to be addressed 



Small-Scale Design 



The small-scale evaluation design fol- 
lows the same basic structure as the large- 



scale design in terms of issues to be ad- 
dressed, variables of importance, and 
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analytic design. However, to reduce the 
scope of the evaluation, seme of the elements 
nust be altered or dropped from the design. 



Data Collection Process 



The data collection processes to be 
eliminated are: 

• The process describing the follow-up of 
participants in the JTPA project will be 
dr ^ped entirely. 

• The observation of teachers using the 
TES A instrument will be dropped entire- 
ly. 

• The qualitative data collection process 
that requires on-site interviews and ob- 
servations will be dropped for the small- 
scale design. 

• On-site monitorl ig and spot checking the 
data collection process will not be pos- 
sible in the small-scale design. 

The data collection process will rely on 
program cooperation and accuracy. The in- 
structions provided with the forms must be 
carefully written and pilot tested. Com- 
munication with the programs through the 
mail and telephone will help the process, but 
some problems should be expected. 

The evaluator in the small-scale evalua- 
tion will use the CAI evaluation form 
provided in Chapter 2 to assess the CAI 
packages in the same manner described in the 
large-scale evaluation. However, depending 



upon resources and time available, the num- 
ber of packages may be restricted. A wide 
representation of CAI packages will be im- 
portant. The use of one of the selected CAI 
packages could be a requirement for 
eligibility to participate in the small-scale 
evaluation. This will limit the population of 
JTPA program that will be included in the 
sampling process'. 



Sample Size 



The sample size will be cut in half for the 
small-roale evaluation. This will permit more 
time tc be spent in obtah ing valid data from 
a smaller sample of programs. The use of 
external personnel will be limited. The re- 
quired computer resources will be less, but 
SPSS-X or SAS on a mainframe will still be 
required. 



Data Analysis 



The small-scale evaluation will require 
the same high level of statistical and research 
skills required in the large-scale project. But, 
it will involve less travel and on-site par- 
ticipation in the data collection process. The 
level of effort required to conduct the 
analysis and report the finding will be ap- 
proximately the same. The number of cases 
involved will have little effect on the statis- 
tical work. 
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Appendix 



Vendors Providing CAI Materials for Preview 
Panel of Experts 
Technology Consultants 
List of JTPA Programs Interviewed 
List of 70001 and Affiliates/CAI Programs 
Comprehensive Competencies Programs (CCP) with JTPA Funding 



Vendors Providing CM Materials for Preview 



Apple Computer 
20525 Mariani Avenue 
Cupertino, CA 95014 
408-996-1010 

Francesca Venning-Townsley, Contact 

BLS Tutor Systems 

5153 W. Woodmill, Suite 18 

Wilmington, DE 19808 

800-545-7766 

Albert Siegfried, Contact 

Conover Company 
P.O. Box 155 
Omro.WI 54963 
414-685-5707 
Terry Schmitz, Contact 

Control Data 

6003 Executive Blvd. 

Rockville.MD 20852 

301-468-8694 

Paula Johnson, Contact 

EDL 

Box 210726 
Columbia, SC 29221 
800-227-1606 
Barbara Elwood, Contact 

Educational Activities, Inc. 

687 M?lin Road 

Newton Square, PA 19073 

215-356-4924 

Connie Huebner, Contact 



Governor's Remediation Iniative 

School of Education, Winthrop College 

Rock Hill, SC 29733 

803-323-2120 

John R. Rumford, Contact 

Hartley Courseware, Inc. 
133 Bridge Street 
Dimondale, MI 48821 
800-247-1380 

HEC Software 
3471 South 550 West 
Bountiful, UT 84010 
800-295-7054 
Linda Eversole, Contact 

IBM 

111 Erford Road 

Camp Hill, PA 17011 

717-731-3618 

Dale Hochmiller, Contact 

Intechnica 

P. O. Box 30877 

Midwest City, OK 73110 

405-732-0138 

Paul B. Edmonds, Contact 

Mindscape 

3444 Dundee Road 

Northbrook, EL 60062 

800-221-9884 

Unisys Corporation 

2200 Renaissance Blvd, Suite 400 

King of Prussia, PA 19406 

215-278-5300 

Adrienne A. Harris, Contact 



qtr 



84 



Contributing Material 



David Barbee 

U.S. Department of Labor, 

CALS/OFR/ILAB 

200 Constitution Ave., Room N-651 1 
Washington, DC 20210 
202-523-5600 

Ted Buck 

Senior Associate 

National Alliance of Business 

1201 New York Avenue, NW, Tune 700 

Washington, DC 20005 

202-289-2823 

Joe Cooney 

National Alliance of Business 
350 Sansome Street, Suite 1040 
San Franv Isco v CA 94104 
415-391-4061 

Cindy Gnech 

Dept of Labor & Industry 

Bureau of Job Training and Partnership 

Room 1105, 7th & Forster Streets. 

Harrisbatg,PA 17120 

717-783-8944 

Dr. Mamoru Ishikawa 
U.S. Department of Labor 
Office of Strategic Planning & 
Program Development & Training 
Administration 

Room 5631, 200 Constitution Avenue, NW 
Washington, DC 20210 
202-535-0672 



Rhonda Johnson 
C >Director 

Pittsburgh Adult Competency Program 
University of Pittsburgh 
5H29L Forbes Quad. 
Pittsburgh, PA 15260 

Raymond E. Manak 

Manager, Adult Learning Center 

Cuyahoga Community College 

Metropolitan Campus 

2900 Community College Avenue, 

LRC 412 

Cleveland, OH 44115 
Ann Meltzer 

American Society for Training and 
Development 

1630 Duke Street, Box 1443 
Alexandria, VA 22313 
703-683-8100 

Valerie Njie 
Director 

Bidwell Training Center 
1815 Metropolitan Street 
Pittsburgh, PA 15233 
412-323-4000 

Dr. Arnold PacKer 

Senior Research Fellow 

Hudson Institute 

1401 Ford Avenue, Suite 200 

Alexandria, VA 22302 

703-824-2048 
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John Rumford 
Acting Director 

Governor's Remediation Initiative 
Winthrop College 
119 Withers Building 
Rock Hill, SC 29733 
803-324-3176 

R eviewing Material 

Charles Blaschke 
President 

Education TURNKEY Systems, Inc. 
256 N. Washington Street 
Falls Church, VA 22046 
703-536-2310 

William Delaney 

U. S. Department of Labor 

Office of Strategic Planning & 

Program Development & Training 

Administration 

Room 5631 

200 Constitution Avenue, NW 
Washington, DC 20210 
202-535-0672 

Jose Figueroa 

National Governor's Association 
444 N. Capitol Street . 
Washington, D<" 20001 
202-624-5394 

Richard Helpin 

Model Comprehensive Education for 
Employment Programs 
P.O. Box 1922 
Austin, TX 78767-1922 
512-472-8220 



Dr. Karen B. Holden 
Executive Director, Job Training 
Glendale Community College - JTPA 
1500 Verdugo Road 
Glendale, CA 91208 
818-240-1000 

Ceci'vj Morales 
Managing Editor 

Employment and Training Reporter 
121 1 Connecticut Avenue, NW 
Washington, DC 20036 

James Parker 
Specialist 

U.S. Department of Education 
Div. of Adult Education 
268 Ridge Road 
Greenbelt,MD 20770 

202- 732-2399 

Paul J. Poledink 

UAW-GM Human Resource Center 
Auburn Hills, MI 48057 

Ira Mozille 
Education Advisor 
Aetna Institute for Corp. Ed. 
15 1 Farmington Avenue 
Hartford, CT 06156 

203- 727-4200 

Tony Sarmiento 

National Coordinator 

Human Resources Dev. Ins. AFL/CIO 

815 16th Street, NW 

Washington, DC 20006 
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Edmund G. Senesac 
Connecticut Bank & Trust 
One Constitution Plaza 
Hartford, CT 06115 
203-244-6628 



Lori Strumpf 

Strumpf Associates 

1015 15th Street, NW, Suite 600 

Washington, DC 20005 
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Technology Consultants 



Contributing Materials 

Donald O. Egner 
Region 111 Project Director 
5806 Pine Hill Drive 
White March, MD 21162 
301-355-2060(H) 

June E. Eiselstein 

Project Consultant, Region 1 and Director 
New Britain, CT 06051 
203-224-3155 or 223-7667 

Ruth Hollenbeck 

Project Consultant, Region VI & VII and 

Director, ABE 

Moberly Area Junior College 

College & Rollins 

Moberly, MO 65270 

816-263-4110 

Lucy Tribble MacDonald 
Project Cc sultant, Region X 
Chemeketa Community College 
P.O. 14007 
Salem, OR 97309 
503-399-5093 

Sheila Shaw 

Project Consultant, Region Sunshine, & 

ABE Resource Teacher 

San Diego Community College 

5350 University Avenue 

San Diego, CA 92105 

619-230-2144 



Reviewing Materials 
Mike KU1 

ALT Technology Consultant, Region Vlll 
Blackfeet Community College 
P.O. 819 

Browning, MT 59417 
406-338-5441 

Christina M. Jagger 
Project Consultant, Region 11 
New Readers Press, LL1 
1320 Jamesville Ave. 
Syracuse, NY 13210 
315-422-9121 

Judith Lipscomb 

Technology Consultant, Region IV 
Mississippi Office for Literacy 
2000 Walter Sillers Bldg. 
500 High Street 
Jackson, MS 39201 
601-359-2789 

Deborah Young 

Project Consultant, Region V, and Director 
UAW-Ford/EMU Academy, Computer 
Project 

7164Carr.slot Drive 
Canton, MI 48187 
313-487-487-6138 

Annabelle Lavier 
Coordinator, ABS Tech. Project 
Treaty Oak Community College 
300 E. 47th Street 
The Dalles, OR 97058 
503-296-6182 
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Lis t ofJTPA Programs Interviewed 

Interviews Site Visits 



MarijaFine 

70001 Training and Employment Institute 
501 School Street SW 
Washington, DC 20024 
800-424-9114 

Maxine Russman 
Partners on Job Training 
1504 3rd Avenue 
Rock Island, IL 61201 
309-793-5261 



BLS Tutor Systems 
5153 W.Woodmill, Suite 18 
Wilmington, DE 19808 
800-545-7766 

Albert & Brad ^egfried, Owners 

Development Center for Adults 

Centre County Vocational technical School 

Pleasant Gap, PA 16823 

814-359-3069 

Marilyn Gentzler, Director 



Richard Gates aid Ed McGee 
2013 West 3rd Street 
Dayton, OH 45417 
513-262-2860 

Laura B. Anderson 

New Castle County Learning Center 

608 Market Street Mall 

Wilmington, DE 19801 

302-654-2215 

Lauretta Furgeson 

One Kennedy Square, Suite 2226 
Detroit, Michigan 48226 
313-963-2022 

Bob Gilbert 
Boise 70001 
911 Mountain Cove 
Boise, ID 83702 
208-338-3618 



Project A.B.E. 
The Yorktowne Mall 
131 N. Duke St. 
York, PA 17401 
717-854-4154 
Dot Foiry, Director 

Project Search 
Gettysburg, PA 17325 
717-334-0221 
Angela Rocks, Director 

Sitco Learning Center 
2107 N. 6th Street 
Harrisburg, PA 
717-236-7931 
Kay Barber 

Tom Skinner Associates. 
272Cha r ^dor Avenue 
Newark, NJ 07112 . 
201-926-4417 
Lauren Skinner, Director 
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List o*70001 and Affiliates/CAI Programs 



70001 Training and Employment 
Institute 

501 School Street SW 
Washington, DC 20024 
800-424-9114 

Marija Fine, Contact Person 



70001 Programs 

Both Comprehensive Competencies 
Program (CCP) and Ideal Learning 

Brendalyn High 
Manager 

Birmingham 70001 
2330 Second Avenue, South 
Birmingham, AL 35203 
205-324-5055 

Evonne Young 
Manager 
Dallas 70001 
309 South Pearl 
Dallas, TX 75201 
214-744-3610 

Peggy Landen 
Manager 

San Joaquin 70001 

807 N. San Joaquin Street, Suite 107 

Stockton, CA 95202 

209-465-5627 



70001 Programs 

CCP or Ideal Learning Only 

David Hawkins 
Manager 

Harris County, TX 
713-987-8505 

Gladys Emerson 
Manager 

70001 Learning Opportunities Center 

1617 Park Place 

Fort Worth, TX 76110 

817-923-5627 

Patricia Foster 

Manager 

Richmond 70001 

6 North 6th Street, Suite 402 

Richmond, VA 23219 

804-644-8132 

Arturo Suarez 

Manager 

302 South Flores 

San Antonio, TX 78204 

512-299-1025 

Cheryl Jennifer 
Manager 

70001 Work and Learning Center 

311 -68th Place 

Seat Pleasant, MD 20743 

301-350-9200 
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70001 Affiliates 

Miscellaneous CAI Programs 

Bob Gilbert 

Manager 

Boise 70001 

911 Mountain Cove 

Boise, ID 83702 

208- 338-3618 

Carolyn Callahan 
Manager 
C-SET 
Stockton, CA 

209- 732-4194 

Bill Kaiser 
Principal 
Bartow County 

Cass Educational Services Center 
Cass High School 
738 Grassdale Road 
Carterville, GA 30120 
404-382-0230 

Jim Smith, Director 
70001 of Indianapolis 
11 South Meridian Street 
Suite 400E 

Indianapolis, IN 46204 
317-633-7000 



ERIC 



Diane Gordon 
Manager 

Licking County 70001 
743 East Main Street 
Newark, OH 43055 
614-345-8659 



Marie Crider 
Manager 

Muskingum County Project 70001 
309-311 Main Street 
Zanesville, OH 43735 
614-454-6212 

Kathleen Mikula 
Manager 

Northwest Ohio 70001 
1933 E. 2nd Street, Plaza Center 
Defiance, OH 43512 
419784-2150 

Larry Miller 
Manager 

Noxubee Educational Services Center 

Noxubee High School 

Highway 14 

P.O. Box 540 

Macon, IAS 39341 

601-726-4225 

Mica Burden 
Manager 

Townsend Center 70001 of Richmond 
855 N. 12th Street 
Richmond, IN 47374 
317-962-7591 
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Comprehensive Competencies Programs (CCP) with 

JTPA Funding 



U.S. Basics Skills Investment 
Corporation 

1700 Diagonal Road, Suite 400 
Alexandria, VA 22314 

202- 667-6091 

Bernadette Martin, Contact Person 

Kristen Phillips 

Institute for Business 

322 East Willamette Avenue 

Colorado Springs, CO 80903 

719-578-550 

Rosanne.Singer 
CREC Work and Lean Center 
11 Asylum Street 
Hartford, CN 06103 

203- 522-9533 

Laura B. Anderson 

New Castle County Learning Center 

608 Market Street Mall 

Wilmington, DE 19801 

302-654-2215 

Jo Ellen Brackett 

New Castle County Learning Center 
3301 Drummond Plaza 
Newark, DE 19711 
302-368-0318 

Joan Hayes 

Alfred E. Beach High School 
3001 Hopkins Street 
Savannah, GA 31401 
912-651-7326 



Georgia Roberson 

Moline Computerized Learning Center 
3430 23rd Avenue 
Moline, IL 61265 
309-797-7901/309-793-5269 

Dennis Brown 

Aledo Computerized Learning Center 
1505 NE 3rd Street, Rt 94 North 
Aledo, IL 61231 
309-793-5232 

Nancy James 

Kewanee Computerized Learning Center 
513 Elliott Street 
Kewanee, IL 61443 
309-852-6544/6545 

Cheryl Smith 

Rock Island Computerized Learning Center 
1504 Third Avenue 
Rock Island, IL 61201 
309-793-5243 

Lucy Murphy 

The Center for Academic Enrichment 
363 N. Water Street 
Decatur, IL 62523 
217-428-5701 

Sheila Conrad 

Business Employment Skills Team, Inc. 
7 Danny Drive 
Streator, IL 61364 
815-672-0699 
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Bob Hoffman 
Connersville School 
1300 Spartan Drive 
Connersville, IN 47331 
3177^825-0521 

Amy Orr 

St Mary of the Woods College 
St. Mary of the Woods, IN 47876 
812-535-3009 

Eileen A. Cruz 
Office for Job Partnerships 
181 Hillman Street 
New Bedford, MA 02740 
508-999-3161 

Geraldine Kerins 

Cape Cod and the Islands Learning Center 
77 High School Road Extension 
Hyannis, MA 02601 
508-771-0141 

Trish Freeh 

Training & Development Corportation 
117 Broadway 
Bangor, ME 04401 
207-945-9431 

J. Trembly 

Training & Development Corportation 
P.O. Box 476 
Ellsworth, ME 04605 
207-667-7543 

Mary Hawkins Warren 
Mountain Valley Training 
P.O. Box G, South A'-enue Station 
Lewistown, ME 04240 
207-795-4060 



Ann Fish 

Career Development Center 
Station 123, 43 Melville Street 
Augusta, ME 04333 
207-289-2006 

Teresa Mudgett 

Training & Development Corporation 
P.O. Box 246 Union Square 
Dover-Folxcroft, ME 04426 
207-564-84-38 

Lauretta Furgeson 
CALX. 

One Kennedy Square, Suite 2226 
Detroit, MI 48226 
313-963-2022 

Bob Johnson 

Woodland Alternative Education Center 
NE 3rd Street 
Staples, MN 56479 
218-894-2439 

Vijya Campagne 

Private Industry Council Learning Center 
50 Broad Street, 6th Floor 
N.Y..N.Y. 10004 
212-425-5525/742-1000 

Jean Stuckert 

Private Industry Council t Inc. 
334 North Sandusky Avenue 
Bucyrus.OH 44820 
419-562-0100 

Kathy Rice 

Private Industry Council, Inc. 
196 South Prospect Street 
Marion, OH 43302 
614-383-2828 
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South Education Center 
1 .055 East Avenue 
Akron, OH 44307 
216-434-9057 

Teresa Jimison 

Cincinnati Institute for Career Alternatives 
19 West Elder Street 
Cincinnati, OH 45210 
513-352-4714 

Karen Wilson 

Portage Private Industry Council, Inc. 
449 South Meridian Street 
Ravenna, OH 44266 
216-297-0720 

Peggy Farison 
Putnam County JTPA 
1307 East Fourth Street 
Ottawa, OH 45875 
419-523-4265 

Cyndee DeLong 
Van Wert County JTPA 
704 East Central 
Van T ^ert,OH 45891 
419-238-5557 

Paula Savage 
Fulton County JTPA 
124 Depot Street 
Wauseon, OH 43567 
419-337-1170 

James Fischbach 
Henry County JTPA 
104 East Washington 
Napoleon, OH 43545 
419-784-2150 
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Toni Jensen 
Williams County JTPA 
P.O. Box 702 
Bry?n, OH 43506 
419-636-6122 

Bob Mayer 

Pauldinf County JTPA 
315 North Walnut Street 
Paulding, OH 45879 
419-399-3345 

Jane Luther 

Treasure Valley Community College 
650 College Boulevard 
Ontario, OR 97914 
503-889-6493 

Malcolm Munson 
Linkville Academy 
220 Pine Street 
Klamath Falls, OR 97601 
J03-882-5691 

EdCink 

All American Family Learning Center 
8 North "F" Street 
Lakeview, OR 97630 
503-947-4883 

JerraKitzmiller 

Permian Basin Learning Center 
1111 West 12th Street 
Odessa, TX 79762 
915-337-1882 

Rosalinda Estrada 

Creative Rapid Learning Center 

408 Congress Avenue 

Austin, TX 78701 

512-472-8220 
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Cheryl Newton 

The Learning Connection - Tyler 
2440 East Fifth Street, Suite 102 
Tyler, TX 75701 
214-597-8131 

TriciaDurrett 

Interactive Systems Laboratories 

P.O. Box 1409 

San Marcos, TX 78667 

512-255-6636 

Alfred Rodriquez 
SANYO 

527 South Main Avenue 
San Antonio, TX 78204 
512-224-6331 

Cheryl Newton 

The Learning Connection - Palestine 
P.O. Box 1460 
Palestine, TX 75801 
214-723-2114 

Glenda Hanners 

Learning Opportunity Center 

1617 Park Place 

Fort Worth, TX 76104 

817-923-5627 



Anna De La Rosa 

The Learning Center 

501 West Thomas Boulevard 

Port Arthur, TX 77640 

409-985-2535 

Joe Martinez 

Concho Valley 

4950 Knickerbocker Road 

SanAngelo.TX 76904 

915-944-9666 

Peggy Cummings 
Abilene Learning Center 
1025 East North 10th Street 
Abilene, TX 79604 
915-672-8544 

Billy Fricks 

Private Industry Council CCP 
111 West Wilson 
Aransas Pass, TX 78336 
512-758-0092 

Dr. Gary Lowe 
Texas A & I University 
Eckhardt Hall #132 
Kingsville, TX 7C36J 
512-595-3991 
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